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Abstract 


The focus of the Synthetic Vision Systems General Aviation (SVS-GA) element of the National 
Aeronautics and Space Administration’s (NASA’s) Aviation Safety and Security Program (AvSSP) was 
to develop technology to help eliminate low visibility induced general aviation (GA) accidents through 
the application of synthetic vision techniques. SVS displays present computer generated three- 
dimensional imagery of the surrounding terrain to greatly enhance pilot's situation awareness (SA), 
potentially reducing or eliminating controlled flight into terrain (CFIT), as well as low-visibility loss of 
control (LVLOC) accidents. 

A critical component of SVS displays is the appropriate presentation of terrain to the pilot. At the time 
of this study, the relationship between the complexity of the terrain presentation and resulting 
enhancements of pilot SA and pilot performance had been largely undefined. The terrain portrayal for 
SVS head-down displays (TP-HDD) simulation examined the effects of two primary elements of terrain 
portrayal on the primary flight display (PFD): variations of digital elevation model (DEM) resolution and 
terrain texturing. Variations in DEM resolution ranged from sparsely spaced (30 arc-sec) to very closely 
spaced data (1 arc-sec). Variations in texture involved three primary methods: constant color, elevation- 
based generic, and photo-realistic, along with a secondary depth cue enhancer in the form of a fishnet grid 
overlay. 

The TP-HDD simulation experiment addressed multiple objectives involving twelve display concepts 
(two baseline concepts without terrain and ten SVS variations), four evaluation maneuvers (two en route 
and one approach maneuver, plus a rare-event scenario), and three pilot group classifications. The TP- 
HDD simulation was conducted in the NASA Langley Research Center’s (LaRC’s) General Aviation 
Workstation (GAWS) facility. The results from this simulation establish the relationship between terrain 
portrayal fidelity and pilot situation awareness, workload, stress, and performance. 

Introduction 

General Aviation (GA) aircraft comprise 85 percent of the total number of civil aircraft in the United 
States of America (USA). In 1999, GA accounted for 85 percent of all accidents and 65 percent of all 
fatalities [1]. The combination of night and instrument meteorological conditions (IMC) increased the 
proportion of fatal to total accidents to 64.3 percent, making it the most deadly GA flight environment. 

The ability of a pilot to ascertain critical information through visual perception of the outside 
environment is limited by various weather phenomena, such as rain, fog, and snow. Since the beginning 
of flight, the aviation industry has continuously developed various devices to overcome low-visibility 
issues, such as attitude indicators, radio navigation, and instrument landing systems (ILS). Recent 
developments include moving map displays, incoiporating advances in navigational accuracies from the 
Global Positioning System (GPS), and enhanced ground proximity warning systems (EGPWS). 
However, all of the aircraft information display concepts (DCs) available to date require the pilot to 
perform various additional levels of mental model development and maintenance and information 
decoding in a real-time environment when outside visibility is restricted [2], 

Synthetic vision systems (SVS) technology will allow this visibility problem to be solved with an 
intuitive, visual solution resulting from better pilot situation awareness (SA) provided by SV displays 
during low visibility conditions, potentially reducing or eliminating controlled flight into terrain (CFIT), 
as well as low-visibility loss of control (LVLOC) accidents. These displays employ computer-generated 
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terrain imagery to present three-dimensional, perspective, out-the-window (OTW) scenes with sufficient 
information and realism to potentially enable operations equivalent to those of a bright, clear day, 
regardless of the outside weather condition [2 through 15]. 

An essential component of all SVS displays is the display of the synthetic terrain on a primary flight 
display (PFD). SVS terrain provides a continuous view of the environment outside the aircraft that gives 
information to the pilot regarding the outside world. SVS terrain also serves as the backdrop for 
integration of the other elements of the display (such as flight data information, guidance symbology, 
etc.). Effective terrain presentation, that conveys the optimum information to the pilot with the lowest 
mental workload, is paramount to successful SVS development and implementation. 

SVS displays, as defined for this study, involve integrated flight information with terrain presented on 
PFD, dual displays (a PFD in conjunction with a navigational display (ND)), and, in the case of a specific 
flight path, such as an 1LS approach maneuver, highway-in-the-sky (HITS) guidance via computer- 
generated predetermined flight routes (i.e., tunnel guidance). The costs of the various components that 
make up SVS displays are decreasing to a point that could make SVS displays fairly affordable, 
especially if associated with adequate increases in safety and operational benefits. 

Current PFDs convey attitude information to the pilot through the use of symbology that has been 
developed and refined over several decades. Specifically, pitch information is provided through a pitch 
scale with a reference waterline symbol and ha nk information is provided via a roll scale. In general, 
pitch scales display approximately 60° of pitch attitude and roll scales are tailored to meet the specific 
needs of the aircraft for which they are designed. 

In the process of creating SVS displays, the computer-generated terrain is integrated with the 
symbology. One part of the integration is the matching of the vertical field-of-view (FOV) of the SVS 
imagery with the pitch scale, which leads to minification of the SVS terrain imagery. Field-of-view is a 
design parameter for SVS PFDs, and as employed in this report, refers to the horizontal FOV of the SVS 
image being displayed to the pilot, unless otherwise noted. Vertical and horizontal FOVs are related 
through the display devices aspect ratio (e.g., 3:4). Variations in FOV have been studied [2] with results 
suggesting that different phases of flight may affect optimum FOVs. 

Numerous publications [2 through 14] are available describing various terrain depiction techniques for 
tactical displays (PFDs, heads-up displays (HUDs)) and strategic ND and multi-function displays 
(MFDs). These techniques include, but are not limited to, ridge lines, grid patterns (equal and non-equal 
spacing), color-coded contour lines, varying color textures based on elevation, photorealistic textures, and 
textures with an embedded grid pattern. Textures increase terrain realism by increasing the level of detail 
per polygon, thus providing additional cues for position and closure rate (height and range) estimates. 
Flight tests have demonstrated that adding a textured terrain skin to electronic attitude display indicators 
(EADIs) and PFDs gave pilots a better awareness of their height above the ground [15]. However, 
references 2 through 12 did not comprehensively investigate terrain portrayal techniques as applied to 
SVS displays, providing only information for specific cases, with limited comparisons. 

Several aspects of SVS terrain portrayal are important. However, the primary elements of SVS terrain 
portrayal are the method of terrain texturing (i.e. wireframe, constant-color, elevation-based generic, 
photo-realistic, etc), and the underlying digital elevation model (DEM). Several DEM resolution options 
are available for application to SVS displays, ranging from low- to high-resolution. Higher resolution 
DEMs (i.e., 1 arc-sec) can generate smaller polygons that can provide a higher fidelity computer- 
generated image. Lower resolution DEMs (i.e., 30 arc-sec) can only generate larger polygons that in 
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some cases could lead to an SVS image appearing different than the actual scene. Prior to this study, 
there was no information relating to the effects of DEM resolution and terrain texturing on pilot SA, 
workload, and pilot performance directly applied to SVS displays. 

Research at the University of Iowa [11] provides detailed information regarding SVS terrain portrayal. 
In this study, a broad spectrum of terrain portrayal techniques were examined using several types of 
experimentation methods, including static and dynamic display evaluations combined with piloted 
simulations of a perspective terrain display located next to an EADI. The objective of this study was to 
establish the minimum effective terrain portrayal technique to maximize the use of certified computer 
platforms with limited capabilities. Reference 1 1 provides a wealth of data regarding human perception 
of SVS terrain portrayal techniques and shows that terrain resolution and texturing significantly impact 
human subjects’ ability to maintain SA on separate displays. However, SVS displays with integrated 
symbology and terrain were not evaluated. 

The terrain portrayal for SVS head-down displays (TP-HDD) simulation was a focused effort that 
evaluated various SVS terrain portrayal methods through careful control of DEM and terrain texture. 
Determining the most advantageous and viable combination of terrain textures and DEM resolution is the 
primary objective of the simulation. Three different types of terrain texturing (constant-color (CC), 
elevation-based generic (EBG), and photo-realistic (PR)), with secondary fish net texturing, and three 
different DEM resolutions (30 arc-sec, 3 arc-sec, and 1 arc-sec) are evaluated in an attempt to develop 
recommendations for terrain portrayal for SVS displays. While higher fidelity (high resolution with the 
advanced terrain texturing) terrain portrayals can be applied to SVS terrain displays, there is an inherent 
increase in SVS system complexity and cost. Previous SVS research has demonstrated that various 
DEM/texturing concepts can work; however proportional benefits of high-fidelity SVS imagery were 
unknown prior to this effort. In-depth research to help define the terrain portrayal requirements for SVS 
displays was needed. The TP-HDD simulation explores these issues to generate critical data that will 
provide a basis of recommendations appropriate for all SVS applications, with a focus on GA. 

HITS guidance symbology on PFDs provides pilots with intuitive information to increase pilot 
performance as well as decrease workload and stress through presentation of the projected desired path 
along with predictive flight-path information, such as velocity vectors. Visual integration of three- 
dimensional (3-D) pathways with SVS terrain provides enhanced SA, with the HITS information 
augmenting the terrain information presented on the PFD. Integration of advanced symbology, such as 
HITS, with SVS terrain was investigated as part of this effort. Assessment of the pilot's ability to fly 
precision approaches with tunnel guidance was also a part of the TP-HDD simulation effort. 

The TP-HDD simulation was conducted to address several critical aspects of SVS displays. The 
objectives of the TP-HDD test series were to: 1) determine the effect of terrain texturing on SA and pilot 
performance for SVS PFDs; 2) determine the effect of DEM resolution on SA and pilot performance for 
SVS PFDs; 3) establish FOV recommendations for SVS PFDs; 4) demonstrate the efficacy of SVS 
displays (for a dual-display configuration (both ND and PFD)) for a comprehensive spectrum of pilots in 
both mountainous and flat-maritime environments; and 5) demonstrate that tunnel guidance symbology 
improves pilot performance, increases SA, and decreases workload. 

Results from this study provide information to help establish the DEM resolution, texturing, and FOV 
requirements for tactical HDDs, based on the phase of flight. This investigation attempts to enhance the 
understanding of SVS displays and quantify their actual benefit, in simulated operations, and to establish 
recommended practices. Another outcome is to demonstrate that non-instrument rated pilots are able to 
fly to an acceptable level of precision, with minimal training, using an SVS-PFD with tunnel symbology. 
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Method 


To address the multiple simulation objectives, a complex experiment involving twelve display 
concepts (two without terrain and ten SVS variations), four evaluation maneuvers (two en route and one 
approach maneuver, plus a rare-event scenario), and three pilot group classifications was conducted. 

Test Platform 

Research Simulator 

The experiment was conducted using the fixed based NASA LaRC General Aviation Work Station 
(GAWS) equipped with a tactical (PFD) and strategic (ND) flight display. The GAWS facility platform 
is based on a modified Precision Flight Control Personal Computer (PC)-based Aviation Training Device 
(PCATD) Model PI- 142 instrument procedure trainer (see figure 1). The model PI- 142 uses hardware 
typical of a GA aircraft with left and right pilot seats. Modifications to the hardware included the 
addition of a 6-in. SVS-PFD, a 15-in. monitor for the baseline round dial (BRD) display, and a United 
Parcel Service Aviation Technologies (UPSAT) MX-20™ MFD. 



Figure 1. NASA LaRC General Aviation Work Station (GAWS) 


The GAWS console was representative of a basic two-crew cockpit environment, with the capability 
of either simulating a single- or twin-engine aircraft. The controls used during the TP-HDD simulation 
were yoke and pedals, power quadrant, and floor-mounted radio stack. In addition, an OTW view was 
projected on a screen beyond the console, to emulate the view from the cockpit window. 
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Three different computers were employed to drive the system in GAWS. One computer, a Pentium-3 
class PC, hosted the Initiative Computing Elite™ Electronic Instrument Flight Rating (IFR) Training 
Environment software which provided the aircraft dynamic responses to pilot control inputs, and control 
of the OTW visibility, as well as data required to generate the research display imagery. Elite™ software 
has the ability to simulate various types of GA aircraft including the generic Cessna- 172 model used in 
this experiment. Two dual-processor, Pentium class CPUs with 2-Gigabytes of RAM and high 
performance graphics cards generated the OTW and the HDD synthetic vision imagery. This SVS 
display software included the underlying databases with the various DEM resolutions and texturing 
concepts, combined with the SVS symbology. 

Display Devices 

The GAWS simulator was designed with a modular instrument panel configuration. For this 
experiment, the subject pilot flew from the left side. The SVS primary flight display (SVS-PFD) research 
panel consisted of a commercial off-the-shelf (COTS) 6.4-in. video graphics array (VGA, 640 by 480 
resolution) liquid crystal display (LCD) operating at 133 pixels per inch (ppi) located directly in front of 
the subject pilot. The SVS-PFD operated in VGA mode with 256K colors. This simulator provided the 
capability of supporting various sizes of displays. For the traditional round dial baseline case, the 6.4-in. 
PFD was replaced by a 15-in., 3:4 aspect ratio, flat panel display. This 15-in. research display presented 
seven 3-in. diameter gauges and operated in extended graphics array (XGA) mode (1024 by 768), and 
provided at least 85 ppi. 

A 15-in. display, located on the right side of GAWS, was used for training purposes only and was 
turned off during formal evaluations. This display also had a 3:4 aspect ratio and operated in the XGA 
mode (1024 by 768) providing at least 85 ppi. 

The display located in the center of the console, where the radio stack typically is located, was a 6-in., 
3:4 aspect ratio VGA LCD UPSAT MX-20™ MFD. The update rate of the aircraft position was once per 
second (1 Hz). Actual display content is described in the Strategic Display section of this paper. 

Front Visual 

A front visual scene was projected on a screen (7-ft wide by 6-ft tall) located approximately 6 ft 
beyond the console. For this test, the subject pilot had a horizontal eye-point viewing angle of 
approximately 41° and a vertical pilot viewing angle of around 29°. The highest fidelity photo-realistic 
HDD research terrain database (1 arc-sec) was rendered for the OTW scene. The front visual provided a 
simulated OTW view with XGA resolution and a 30 Hz update rate. The front visual scene was also 
capable of simulating weather to the extent of creating Instrument Meteorological Conditions (IMC) and 
transitioning from Visual Meteorological Conditions (VMC) to IMC, by manipulating variables such as 
current visibility, cloud coverage, and ceiling conditions. 

Field of View Mechanism 

A pilot-selectable display FOV control was developed and available for the subject pilot to use 
throughout the experiment. The four FOV choices for this experiment were 22°, 30°, 60°, and 90°. 
Symbology was changed to remain conformal to the terrain for the various FOVs. Two FOV control 
options were available to the subject pilots. 

The first control option was the rotary knob physically located to the left of the SVS PFD. This rotary 
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knob had four positions. The first three positions provided 22°, 30°, and 60° FOVs. The fourth position 
enabled FOV selection from the yoke button. This control option did not provide a 90° FOV selection to 
the subject pilot. 

The second control option was the yoke button, which was physically positioned on the right horn of 
the yoke. The yoke button, which was disabled when the rotary knob was not in the fourth position, 
provided 22°, 30°, 60°, and 90° FOV options, cyclical one-way selection capability, and hands-on-wheel 
ease of FOV control. 

Physiological Set-Up 

Physiological data were collected to obtain a better understanding of the subject pilot’s physical state 
during the approach and rare-event maneuvers. Three (3) physiological measures, heart rate 
(Electrocardiogram - ECG), skin temperature (SKT), and muscle response (Electromyogram - EMG), 
were monitored and recorded to document subject pilot response during the approach runs of the TP- 
HDD experiment. Sensors were attached to the subject’s skin at locations on the left arm, right ankle, and 
bracketing the heart. For more information regarding the actual set-up of the physiological system, see 
reference 17. 

Participants 

To perform a comprehensive evaluation of SVS technology for the GA community, as well as to study 
the effects of various terrain portrayal concepts on a broad spectrum of pilots of various backgrounds and 
experience levels (ranging from low-hour Visual Flight Rules (VFR) ratings to NASA test pilots), the 
subject pilots were grouped into three categories (see table 1). The first group of subjects consisted of 
fourteen low-time pilots, each with less than 400 hours and no instrument training beyond that required 
for the private pilot’s license rating. The second group was comprised of six IFR-rated pilots with less 
than 1000 hours. The last group of subject pilots consisted of four professional test pilots from NASA 
and the Federal Aviation Administration (FAA) and three Juneau (Alaska) area commercial operators 
with total flight time over 2000 hours. Pilots from Juneau, Alaska, were included in this experiment due 
to their being potential SVS users as part of the FAA Capstone-2 project. 
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Table 1. Subject pilot participant experience summary 


Low-Hour VFR 


Low-Hour IFR 


High-Time 

Subject 

Flight 

Hours 

Subject 

Flight 

Hours 

Subject 

Flight 

Hours 

2 

97.0 

4 

195.0 

1 

6000.0 

3 

113.6 

10 

154.0 

5 

11000.0 

7 

100.0 

12 

385.0 

6 

9850.0 

8 

211.0 

16 

130.0 

11 

4500.0 

9 

50.4 

17 

174.5 

13 

10000.0 

15 

115 

19 

600.0 

14 

2800.0 

18 

80.3 



20 

7000.0 

21 

99.0 





22 

65.0 





23 

174.5 





24 

235.0 





25 

77.0 





26 

59.4 





27 

101.0 











Ave 

112.7 

Ave 

273.1 

Ave 

7307.1 

St Dev 

55.8 

St Dev 

184.3 

St Dev 

3091.7 


Terrain Databases 

The terrain databases developed for the simulated area of operations was that of the relatively 
mountainous region around the Roanoke Regional Airport (FAA airport identifier, ROA) at Roanoke, 
Virginia (VA), defined by the boundaries of 37° 30’N, 79° 40’W, 37° OO’N, and 80° 40’W. These 
terrain databases were combinations of various DEM resolutions (from the ROA region) and terrain 
texturing concepts and were built and rendered by commercial COTS tools. Three-dimensional models of 
airports (including runways with markings), buildings, obstructions, and other man-made structures were 
created and inserted into the databases. The update rate requirement for these terrain databases was 30 
Hz, with a minimum acceptable momentary update rate of 20 Hz. 

Digital Elevation Models (DEMs) 

DEM resolution defines the distance between elevation data points (post-spacing) for a given database. 
For this experiment, the source elevation data were based on the United States Geological Survey (USGS) 
National Elevation Dataset (NED), with an accuracy of within 12 m (90% of data) horizontal and 7 m 
(90% of data) vertical. Three specific DEM resolutions were investigated during the TP-HDD 
experiment to cover a broad range of viable DEM options. The low resolution, 30 arc-sec (900-m (2953- 
ft) post-spacing) DEM (DEM30) was selected because it is freely available and currently used in some 
industry SVS applications due to the low computational power required for rendering. The medium 
resolution, 3 arc-second (90-m (295-ft) post-spacing) DEM (DEM3) was selected since it was also 
becoming relatively available. The highest resolution, 1 arc-sec (30-m (98-ft) post-spacing) DEM 
(DEMI) option was investigated to form an upper bound for current consideration. 
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Figure 2 illustrates, spatially, the differences between DEMI (green dots), DEM3 (blue dots), and 
DEM30 (red dots). 


Red Dots: 30 arc-sec ~ 900 m 
Blue Dots: 3 arc-sec ~ 90 m 
Green Dots: 1 arc-sec ~ 30 m 

Figure 2. Spatial representation of DEM resolution 

The images below are the triangularized irregular network (TIN) polygons, a method of representing 
surfaces efficiently by varying the "closeness" of points according to the amount of surface undulation 
detail they possess, created from each of the three DEM resolutions (figure 3). The images are from the 
same viewpoint in an area near ROA in VA, and the viewpoint elevation was approximately 4250 ft mean 
sea level (MSL) with a 0° pitch attitude for all three images. 



a) 30 arc-sec DEM b) 3 arc-sec DEM c) 1 arc-sec DEM 

Figure 3. DEM comparison from same viewpoint near ROA 

It should be noted that higher resolution databases are much larger in terms of the overall number of 
data points for a given area of coverage with higher storage and computational expenses associated with 
manipulating and rendering these data. In addition, the smallest polygon that can be created with a given 
DEM has sides equal to the distance between data points. For example, the smallest possible polygon 
employed with the 30 arc-sec DEM would have sides 2953 feet long. Since the lower resolution DEMs 
are less populated, substantial terrain features might be excluded. The possibility of losing entire peaks as 
well as detailed terrain relief in the lower resolution databases exists. In figure 4, a rounding distortion 
effect becomes apparent between the DEMI and the DEM30 (refer to the area indicated by the arrows). 
For this example, the EBG texturing concept is employed. 





Rounding Effect 



a) 1 arc-sec DEM b) 30 arc-sec DEM 


Figure 4. Apparent “rounding effect” from DEMI to DEM30 
Terrain-Texturing Concepts 

Terrain-texturing refers to the method used to fill the polygons that comprise the terrain database. The 
three primary texturing concepts tested were CC, EBG, and PR. The CC texturing concept represented an 
industry concept that has completed the process of FAA certification in the Capstone-2 program and 
maintains the “blue sky over brown ground” concept from traditional PFDs. This texturing concept 
requires the least amount of computational resources for rendering (of textures evaluated in this 
experiment), increasing the potential for use of currently certified avionics platforms for SVS 
applications. 

The familiar sectional-like coloring and intuitive characteristics of the EBG texturing concept made 
this terrain texture a candidate for evaluation. This texturing concept consisted of twelve equal-height 
coloring bands that correspond to different absolute terrain elevation levels, similar to the colors 
employed for VFR sectional charts (see table 2). Lower terrain levels are colored with darker colors, 
higher terrain levels are assigned lighter colors. A certain shade of green (Index 0 in table 2) was set to 
the field elevation, 1 176 ft mean sea level (MSL). The lightest color was set to the highest terrain within 
50 nm of ROA, approximately 4000 ft MSL. 
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Table 2. Elevation-based generic texture color band elevations at ROA 


Index 

Color 

Elevation (m) 

Elevation (ft) 

0 |H 

300.00 - 

401.60 

984.25ft- 1317.59ft 

1 

■■ 

401.60 - 

503.20 

1317.59ft- 1650.92ft 

2 


503.20 - 

604.80 

1650.92ft- 1984.25ft 

3 

■■ 

604.80- 

706.40 

1984.25ft -2317.59ft 

4 

wm 

706.40 - 

808.00 

2317.59ft -2650.92ft 

5 

mm 

808.00 • 

909.60 

2650.92ft - 2984.25ft 

6 

mm 

909.60 - 

1011.20 

2984.25ft -3317.59ft 

7 


1011.20- 

1112.80 

3317.59ft -3650.92ft 

8 


1112.80- 

1214.40 

3650.92ft - 3984.25ft 

9 


1214.40- 

1316.00 

3984.25ft -4317.59ft 

10 


1316.00- 

1417.60 

4317.59ft -4650.92ft 

11 


1417.60- 

1519.20 

4650.92ft - 4984.25ft 


The life-like, computationally-taxing PR texturing concept was considered to establish a possible 
upper bound for SV terrain texturing. The PR texturing concept was created by overlaying color 
orthorectified 1 4-m satellite imagery data onto a DEM database. The resulting scene was a realistic view 
of the ROA area. PR texturing requires special graphics hardware because of the amount of texture 
memory required to render the scene in real-time. 

Cultural Feature Data 

For the CC and EBG terrain-textured display concepts, cultural features, such as roads and rivers, were 
included as objects in the terrain database. For the PR concepts, cultural and feature data were supplied 
naturally through the photo-texture images. 

Fish Net Overlay Concept 

In addition to the primary terrain texturing concepts, a fishnet (FN) grid overlay was added to several 
DCs. The theory of the FN grid involves placing grids of known size within the synthetic scene to 
facilitate pilot’s depth perception. The potential benefits of the FN grid are cues for depth perception, 
distance, angular orientation and angular rates. The spacing of the FN overlay was 500 ft by 500 ft, 
regardless of the DEM resolution. The FN grid was dual-color (gray/white) to provide a haloing effect 
and compensate for color variations of features within the terrain databases (e.g., lighter colors of 
populated areas for the PR texture) and was oriented along cardinal headings (north, south, east, west). 


1 According to Jeppesen, the process of removing geometric errors inherent within photography and imagery due to 
relief displacement. 
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Airport Models and Objects 

The ROA airport model included runways with all runway markings, along with the most significant 
airport buildings. Airport buildings were developed to appear like the actual buildings they represented as 
viewed from approximately 3 nm. All models were placed on top of the underlying terrain database. 
Objects/obstacles greater than 200 ft high within 20 nm of ROA were represented by narrow rectangular 
barber-striped pole objects indicating their respective estimated heights and locations. 

Display Types 

The displays evaluated in the TP-HDD experiment were grouped into three types: the baseline round 
dials, the Blue Sky/Brown Ground (BSBG) PFD, and the SVS PFD (see figure 5). The MX-20™ display 
was shown in conjunction with all of the experimental display types. 



a) Baseline Round Dials 


b) Blue Sky /Brown Ground PFD c) Synthetic Vision 


Figure 5. TP-HDD display types 

Baseline Round Dials: 

The BRD display type (figure 5) replicates instrumentation currently found in the vast majority of GA 
aircraft, including airspeed, attitude, altitude, turn coordinator, directional gyro, and vertical speed 
indicators. During approach maneuvers, ILS course deviation indicators were also displayed, along the 
side and bottom of the attitude indicator. 

Blue Sky/Brown Ground PFD: 

The BSBG concept replicated conventional PFDs, featuring integrated information (i.e. airspeed, 
altitude, attitude) into one display. The BSBG PFD symbology (to be detailed further in a subsequent 
section) included items such as a velocity vector with sideslip flag and acceleration caret, air data tapes, 
and horizon line. During approach maneuvers, tunnel guidance symbology was shown in conjunction 
with localizer and glideslope guidance. 

Synthetic Vision Systems PFD: 

The SVS PFD was identical in format to the conventional PFD, with the exception that various SVS 
terrain portrayals replaced the BSBG background. The terrain portrayal concepts were developed from 
combinations of DEM resolutions and texturing methods. 
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Experimental Display Concepts 

In this study, ten different PFD configurations (figure 6 (a-j)) of various texturing concepts and DEM 
resolutions were evaluated to explore terrain portrayal on the PFD: the 1 arc-sec DEM, constant-color 
texture with fish net (CCFN1), and the 30 arc-sec DEM, constant-color with fish net (CCFN30), the 1 
arc-sec DEM elevation-based generic texture (EBG1), the 1 arc-sec elevation-based generic texture with 
fish net (EBGFN1), the 3 arc-sec DEM elevation-based generic texture with fish net (EBGFN3), the 30 
arc-sec DEM elevation based generic texture with fish net (EBGFN30), the 1 arc-sec DEM photo-realistic 
texture (PR1), the 1 arc-sec DEM photo-realistic texture with fish net (PRFN1), the 3 arc-sec DEM photo- 
realistic texture with fish net (PRFN3), the 30 arc-sec DEM photo-realistic texture with fish net 
(PRFN30). While the FN was designed to enhance the EBG and PR primary texturing concepts, it was 
determined to be essential for CC texturing due to the low amount of terrain information visible without 
it. 



c) EBG1 


d) EBGFN1 
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e) EBGFN3 


f) EBGFN30 



g) PR1 


h) PRFN1 



2999R 


2999R 


i) PRFN3 


j) PRFN30 


Figure 6. TP-HDD display concepts 
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Field of View 


As previously stated, field of view (FOV) is a design parameter that has specific importance for SVS 
displays. Larger FOVs permit pilots to view larger areas but require the display image to deviate away 
from a conformal condition. Larger FOVs, while useful during turns or in turbulence, make objects 
appear further away (objects are minified). Variations in FOV affect the pilot’s ability to judge distances. 
Lower FOVs provide an image that becomes more nearly conformal and enhances depth perception 
(objects are less minified). Objects that are narrow, like runways, become more visible with lower FOVs. 

In order for an SVS image to be conformal, objects in the image need to subtend the same angles they 
do in the real world. Conformal SVS displays provide the size, shape, and location of the terrain to the 
pilot exactly as it would appear if the SVS display were a window. The conformal FOV of a display 
device is based on the size of the display device and the distance from the display device to the pilot’s eye 
reference point (ERP), which was 24 in. for this experiment. 

SVS imagery can be generated for almost any FOV and displayed to the pilot. The degree to which 
the SVS imagery deviates from the conformal FOV is referred to as the Minification Factor (MF). The 
MF is defined as the FOV of the imagery being displayed to the pilot divided by the conformal FOV of 
the display device. The MF is also the inverse of the magnification factor. Conformal FOV is also 
referred to as unity magnification/minification. 

Figure 7 presents SVS PFD images for the 30°, 60° and 90° FOVs, for identical aircraft positions, 
approximately 5000 ft above ground level (AGL). A MF of 2.6 resulted for the 30° FOV, the 60° FOV 
produced a MF of 5.3, and the 90° resulted in a MF of 7.9 for this size display. From these images, the 
effect of variations of the MF can be seen. Increased MFs create the illusion that objects (like the 
populated areas) are further away as well as the appearance that the aircraft altitude is decreased, affecting 
the pilot’s ability to judge distances. The capability to change FOVs during particular phases of flight 
provides the pilot with a variety of cues in terms of terrain and symbology sensitivity. While flying each 
of the evaluation tasks, the subject pilots were encouraged to scroll through the FOV options during each 
phase of flight, evaluate the options, and provide comments during this experiment, as well as supply a 
resulting FOV strategy at the completion of the experiment. 
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a) FOV 30° b) FOV 60° 



c) FOV 90° 


Figure 7. FOVs of 30°, 60°, and 90° from same geographic location 

Symbology 

For the BRD concept, the symbology was similar to that of a traditional GA instrument panel: airspeed 
indicator, attitude indicator, altimeter, turn coordinator, directional gyroscope, vertical speed indicator, 
and gauge indicating engine revolutions per minute (RPM). For approach guidance, localizer and glide 
slope information was available along the side and bottom of the attitude indicator. 

On the PFD, symbology for all display concepts featured advanced GA symbology elements (see 
figure 8). 
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Figure 8. Integrated symbology on the PFD 

Air-data information was presented by integrated airspeed and altitude tapes. Airspeed trend 
information was also included in the airspeed tape by a green bar that indicated the expected airspeed in 
10 seconds. A vertical speed indicator was included in the integrated altitude tape. A roll pointer with a 
sideslip wedge and magnetic heading digital read-out, and a pitch ladder provided heading and attitude 
information. Additionally, a velocity vector cluster was present, utilizing a non-quickened velocity vector 
that depicted current aircraft flight path and track angle with an acceleration-along-flight-path indicator 
(off the left finlet of the velocity vector marker). 

Additional symbology was presented for approach maneuvers. Course deviation indicators and a 
tunnel in the sky concept provided guidance information on the PFD. Vertical and lateral dogbone- 
shaped path deviation indicators supplied the pilot with information regarding proximity of the aircraft to 
the center of the tunnel. Diamond-shaped course deviation indicators were provided to show localizer 
and glideslope error. Both the dogbone-shaped path deviation indicators and diamond-shaped 
localizer/glideslope error deviation indicators were co-located on the same scales. 

A tunnel in the sky concept was employed during approach maneuvers for all but one SVS display 
concept (used for a within-concept on/off tunnel comparison) and the BRD concept. The tunnel in the sky 
concept featured a series of unconnected 400 ft wide by 320 ft tall uniform green rectangles depicting the 
desired flight path for the approach scenario, providing most of the lateral and vertical path guidance. 
Tunnel spacing was dependent on FOV. For the wider FOV’s, the tunnel boxes were closer together; for 
the smaller FOV’s, the tunnel boxes were spaced farther apart (i.e.: FOV=90°, distance between boxes 
was 965 ft; FOV=30°, distance between boxes was 4685 ft). During turns, if the required bank angle was 
greater than 5°, the boxes were tilted 20° to cue the turn. This tunnel concept replicated an industry 
application of this technology with SVS terrain. 
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Strategic Display 


Strategic terrain display information was provided by a UPSAT MX-20™ MFD located in the radio 
stack. On the MFD, terrain awareness, route information, waypoints, and towers were portrayed (figure 
9). All display concepts were evaluated in the presence of the MFD. Terrain more than 2000 ft below the 
aircraft was portrayed in black, terrain between 2000 ft and 500 ft was green, terrain between 500 ft and 
ownship altitude was yellow, terrain at or above ownship altitude was red. A GPS unit was used to 
program the flight paths for each of the pre-determined flight routes, which was displayed to the pilots on 
the MFD. 



Figure 9. Strategic display in “Terrain Warning” mode 

Evaluation Tasks 

The evaluation tasks were developed to cover critical phases of flight. To add some additional 
sensation of realism and more representative levels of workload, a low/moderate level of turbulence was 
simulated throughout each run. Figure 10 illustrates the simulation area of operations. The en route 
maneuvers, including the rare-event, are indicated by the red flight path. The blue flight path signifies the 
approach maneuver. The en route maneuvers were designed specifically to take advantage of the 
abundance of terrain on the Poor Mountain ridge line in the ROA local area. 



Figure 10. Simulation area of operations 
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En Route Maneuver 


Two en route maneuvers were flown that required the subject pilot to maintain assigned heading, 
airspeed, and altitude values at different points during a 5-minute task. The en route maneuvers began 19 
nm southwest of ROA, with a heading of 140° and an indicated airspeed of 100 knots (KIAS). The high- 
altitude en route task was initiated at 9500 ft MSL (approximately 7000ft AGL), while the low-altitude en 
route maneuver began at 6500 ft MSL (approximately 4000 ft AGL). For both maneuvers, pilots were 
required to fly straight and level for approximately 2.5 minutes, maintaining heading, airspeed, and 
altitude. With the help of the strategic display to identify a fly-by waypoint, the subject pilots then 
executed a left turn, using 20 degrees of bank, to a heading of 050°, while simultaneously descending 
1500 ft (over rising terrain). For this maneuver, part of the descent took place during the 90° turn, and the 
rest of the descent was completed while maintaining the second target heading. The target level-off 
altitude for the high-altitude task was 8000 ft MSL (approximately 4000 ft AGL), while the target altitude 
for the low-altitude task was 5000 ft MSL (approximately 1000 ft AGL). 

At the starting point of the maneuver, VMC was simulated. One minute into the flight a one-minute 
transition into IMC was simulated by reduction of visibility on the OTW display to one statute mile. 
Subject pilots were asked to vary FOV during the entire maneuver to any desired setting. At the end of 
the maneuver, the subject pilots were asked to cycle through FOVs, one more time, to support their 
evaluations. 

Approach Maneuver 

The approach maneuver consisted of a 6.5-minute evaluation starting with a straight-and-level flight 
on a 30° localizer intercept course for the 1LS 33 approach into ROA. The target indicated airspeed 
throughout the maneuver was 90 knots. The subject pilots were tasked to fly a heading of approximately 
300° to join the localizer (roughly 10 nm from the threshold) and maintain 2640 ft until intercepting the 
glide slope at approximately 4.5 nm, then continue flying the approach to 200 ft AGL (1379 ft MSL). 
This initial altitude provided about a 400 ft clearance over a ridgeline that was traversed on the 1LS 
intercept segment. During runs where the tunnel was present on the PFD, the subject pilots flew the 
tunnel for guidance. OTW visibility was reduced from VMC to one statue mile within the first minute of 
the flight. In addition to moderate turbulence that decreased throughout the run, wind was simulated to be 
from 030° at 15 knots, decreasing to 5 knots on late-final approach. Subject pilots were asked to vary 
FOV during the entire maneuver to any desired setting. 

Rare-Event Maneuver 

‘Rare-event’ simulation techniques require many nominal simulation trials to produce only a few trials 
containing the data of interest. As employed for this effort, rare-event testing attempted to generate high- 
quality data reflecting when pilots were exposed to a completely unexpected event of significant research 
interest. The purpose of this maneuver was to determine if SVS PFD concepts provided terrain SA 
sufficient to avoid CFIT accidents in an unexpected situation. 

The rare-event task simulated a flight scenario with an incorrect altimeter setting. Effectively, the 
altitude tape indicated the incorrect (higher) altitude, which was different from the actual altitude 
portrayed by the terrain on the PFD. In addition, the altitude provided to the MX-20 also included the 
same 1500 ft error. This task was administered as the last run of the data collection for each subject pilot 
and was designed to look like the low-altitude en route task. The subject pilots were not alerted to the 
rare event and thought they were just re-running a previous test condition. 
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The rare-event maneuver started at the same position as the low-altitude en route task, but at an 
altitude 1500 ft lower. Consequently, the target level-off altitude was 500 ft below several mountaintops 
directly in front of the aircraft. The weather state was the same as during the low-altitude en route task, 
with visibility degrading one minute into the run. Display concepts (excluding baseline concepts) were 
randomized among pilots repeating one of the display concepts already flown. The baseline case was not 
evaluated for this scenario since the pilots would have absolutely no cues at all to indicate the correct 
altitude. 


Experiment Design 

Test Matrix 


Combinations of DEM and texturing concepts (including the FN option) produced 18 different 
potential terrain portrayal concepts to evaluate. A preliminary look at these 18 display concepts revealed 
that the CC texture without a FN provided little evident information on terrain contouring. As a result, 
the CC texture without the FN was eliminated as a viable concept for terrain portrayal. For the remaining 
15 concepts, a usability study was then conducted in order to down-select a smaller and more manageable 
set of candidate display combinations for more thorough investigation. While the formation of this set of 
candidate display concepts was based on subject pilot evaluations, the suggested collection of display 
concepts was broad enough to meet the objectives of the experiment. 

A total of 10 SYS DCs were evaluated for this simulation experiment (table 3). 


Table 3. Display concepts experimental matrix 


Texturing Concepts 



EBG 

PR 

CCFN 

EBGFN 

PRFN 

30 



V 

V 

V 

3 




V 

V 

1 

V 

V 

V 

V 

V 


These 10 SVS display concepts were evaluated in addition to the specific baseline concept flight 
displays discussed earlier. These two baseline flight displays, BSBG and BRD were split evenly between 
subject pilots within each distinct pilot group. In addition to display type variations, for the approach 
maneuver a tunnel-off condition was evaluated for the CCFN texture with the 30 arc-sec DEM concept 
(CCFN30nt, for “no tunnel”). All display concepts were randomized among the pilots, for each 
maneuver. 

Evaluation maneuvers were blocked (high- altitude en route, low-altitude en route, or approach) with 
DCs being randomized to counter pilot variability and learning and fatigue effects. The high-altitude en 
route block was always conducted first, then the low-altitude en route block, followed by the approach 
block. The rare-event maneuver was conducted at the end of the approach block. Under the assumption 
that the baseline concepts (i.e., BRD and BSBG) would always result in a hazardous situation since the 
pilots had no cues to indicate the correct altitude, they were not included in the rare-event scenario. Table 
4 represents the run list for the first subject pilot. The complete run list for all subject pilots can be found 
in Appendix A. 
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Table 4. Run list for subject pilot 1 


Subject 1 | 

Block 1: High Altitude 

Runl 

PRFN3 

Run2 

CCFN1 

Run3 

BSBG BL 

Run4 

EBG1 

Run5 

EBGFN3 

Run6 

PR1 

Run7 

PRFN30 

Run8 

EBGFN30 

Run9 

PRFN1 

RunlO 

EBGFN1 

Run I t 

CCFN30 

| Block 2: Low Altitude 

Runl2 

BSBG BL 

Run 13 

EBGFN3 

Runl4 

PRFN30 

Runl 5 

CCFN1 

Run 16 

PR1 

Runl 7 

EBGFN1 

Runl 8 

PRFN3 

Runl9 

CCFN30 

Run20 

EBGFN30 

Run21 

PRFN1 

Run22 

EBG1 

| Block 3: Approach 

Run23 

PRFN1 

Run24 

CCFN30 

Run25 

EBGFN3 

Run26 

BSBG BL 

Run27 

PR1 

Run28 

EBG1 

Run29 

EBGFN30 

Run30 

CCFN30nt 

Run31 

EBGFN1 

Run32 

CCFN1 

Run33 

PRFN30 

Run34 

PRFN3 

Rare Event 

Run35 | CCFN1 


Simulation Operations 

The experiment was conducted during a 2.5-month period of time with no substantial schedule 
interruptions. Each pilot participated in approximately two days of testing, consisting of 35 trial runs. 
Before the start of the experiment, each pilot received an extensive pilot briefing, as well as 
approximately one-hour of training time in the GAWS with a FAA certified flight instructor for 
instruments (CFII). The goals of these briefings and training were to familiarize each subject with the 
objectives of the experiment and educate the subjects on the salient features of the symbology and 
simulator functionality. 

Eleven trial runs were performed for the high-altitude and low-altitude en route maneuver blocks, with 
each run being approximately 5 min. Twelve trial runs were performed for the approach block 
(approximately 6.25 min per run), including the tunnel off CCFN30 run (CCFN30nt). Subject pilots were 
informed as to which DC they were evaluating prior to each run. As previously stated, the single rare- 
event maneuver was typically the last run of the experiment for each subject pilot, with the exception of 
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one subject pilot, to counter previous exposure to similar experiment design where rare-event testing was 
utilized. A total of 945 trial runs were accumulated. 

The subject pilots completed subjective questionnaires after each run, after each block, and at the 
conclusion of the simulation. All subjects were encouraged throughout the experiment to provide verbal 
feedback on all aspects of the test. Subject pilot comments were recorded real-time during each run, after 
each block, and during the final semi-structured interview, via the audio and video recordings. 

Dependent Measures 

Several subjective measures were recorded to estimate levels of workload and situation awareness 
during the TP-HDD experiment. Over 216 hours of multi-media recordings were compiled. The 
multimedia recordings allow the researchers to determine what each subject pilot was experiencing, real- 
time, while dynamically working with each DC. All subject pilot comments and remarks were 
transcribed and analyzed. A condensed version of the transcriptions is readily available on CDROM as a 
supplement to this report. In addition to subjective measures, performance data such as flight path error 
and pilot control activity were also recorded and analyzed. A statistical software package, Statistical 
Package for the Social Sciences (SPSS) which provides predictive analytics, was used to statistically 
analyze the data, using a 5-percent statistical level of significance (p-factor). 

Qualitative Data 

Questionnaires were administered to collect qualitative data using different techniques, ratings, and 
questions to solicit specific information from each subject pilot regarding SA, mental and physical 
workload, terrain awareness, and preferences. 

Run Questionnaire: 

After each run, the subject pilot filled out a post-run questionnaire, based on the DC that was utilized 
for that particular run (see Appendix B). The questionnaire was designed to illicit specific types of 
information regarding workload and SA. Along with the NASA Task Load Index (TLX) (total workload) 
and a Situation Awareness Rating Technique (SART) (an SA subjective measure), questions relating to 
stress and terrain awareness were also posed. 

Block Questionnaire: 

After each maneuver block, a block questionnaire was administered (see Appendix C), specific to the 
baseline DC that each pilot was assigned. This questionnaire consisted of the SA-Subjective WORkload 
Dominance (SWORD) Technique [17], text questions regarding subject pilot FOV strategy, performance 
and terrain awareness ratings for each display concept, and general questions about the use of the primary 
and strategic flight displays. Using the SA-SWORD, the subject pilots were asked to compare one terrain 
texturing concept versus another, in a pair-wise comparison (across all terrain-portrayal concepts), using a 
nine-point scale. These comparisons, for each pair of display concepts, indicate the subject pilot’s 
determination of the level of SA enhancement one DC may provide over another DC. Due to the sheer 
size of the SA-SWORD when applied to the test matrix, the SA-SWORD only included terrain-texture 
concepts combined with the multiple DEM resolutions. Neither the baseline display concepts, nor the 
tunnel-off condition, were evaluated with the SA-SWORD measure. 
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Exit Interview: 


At the completion of each subject pilot’s experiment, an exit interview was conducted, also specific to 
the pilot’s respective baseline (see Appendix D). The interviews focused on verbal and visual protocols 
to elicit participant responses, including texture, DEM, and DC preferences (in general and specific to 
maneuver), FOV strategy, and subject opinion on usefulness and benefit of each DC during specific 
scenarios (such as CFIT). These interviews were semi-structured, with flexibility to draw upon comments 
from each facet of the experiment. 

Quantitative Data 

Performance data, such as lateral and vertical path error, glide slope and localizer error, altitude, 
heading, bank angle, and airspeed errors, and pilot control input data, were also collected and analyzed 
during the experiment. Root mean square (RMS) of the above metrics were computed, based on 
maneuver, and analyzed by Analysis of Variance (ANOVA) across display type and terrain texture. If a 
statistically significant effect was discovered by the ANOVA, a Student-Newman-Keuls (SNK) post-hoc 
test of individual means was performed to further investigate the data. 

Performance levels were established that represent desirable levels of pilot control of the aircraft, and 
the percent of the time that the pilots maintained desired (Fevel 1), adequate (Fevel 2), and below- 
adequate performance was analyzed. These performance levels are based on a variation of the FAA 
Instrument Rating Practical Test Standards (PTS) [18] metrics. 

Desired performance (Fevel 1) was defined as maintaining the following parameters (as appropriate 
for each maneuver): 

Airspeed error <±10 knots 
Altitude error < ± 1 00 ft 
Heading error <±10° 

Bank Angle error < ± 10° 

Focalizer error < ± 1 dot (During approach, only) 

Fateral path error < ± 1 dot (During approach, only) 

Glide slope error < ± 1 dot (During approach, only) 

Vertical path error < ± 1 dot (During approach, only) 

Adequate performance (Fevel 2) was characterized as maintaining the following parameters (as 
appropriate for each maneuver) during the experimental run: 

Airspeed error between ±10 and ± 20 knots 
Altitude error between ±100 and ± 200 ft 
Heading error between ±10° and ± 20° 

Ba nk Angle error between ±10° and ± 20° 

Focalizer error between ± 1 dot and ± 2 dots (During approach, only) 

Fateral path error between ± 1 dot and ± 2 dots (During approach, only) 

Glide slope error between ± 1 dot and ± 2 dots (During approach, only) 

Vertical path error between ± 1 dot and ± 2 dots (During approach, only) 

Below-adequate performance (Beyond Fevel 2) was identified as flying outside of the aforementioned 
adequate performance parameters for the majority of the experimental run (beyond twice PTS). 
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For the duration of this experiment, in addition to the parameters mentioned above, other factors were 
recorded such as FOV settings throughout each run and various pilot control activity elements, such as 
pitch (longitudinal input), roll (lateral input), and yaw (directional input) control, along with throttle 
settings. The maximum deflection values for the simulator in the TP-HDD configuration are presented in 
table 5. Control displacements were commensurate with this type of training facility. 


Table 5. Maximum deflection values for Pilot Control Activity parameters 



Longitudinal 
(+left, -right) 

Lateral 
(-left, +right) 

Directional 
(-left, +right) 

Throttle (Idle = 0) 

Max 

Deflections 

-84.15 

-89.1 

-69.3 

0 

(counts) 

84.15 

89.1 

68.1 

86.52 


En Route Measures 

For data analyses, all En Route runs were separated into seven segments: straight and level (constant 
heading); initiate turn and descent; constant turn and descent; roll-out and continue descent; constant 
heading descent; stop descent (constant heading); and straight-and-level at target altitude (constant 
heading). In addition to analysis of the entire length of the run as a whole, these seven segments were 
prioritized for data reduction and analyses. Figure 1 1 illustrates each segment during the en route 
maneuver. 


Segment 7: Straight and Level at Target Altitude 



Figure 11. Top view of the en route maneuver segments 

The average run time for the en route maneuver was 5 min. The segment averages specific to the en 
route maneuver are detailed in table 6. 
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Table 6. Segment duration averages for the en route maneuvers 



Seg. 1 

Seg. 2 

Seg. 3 

Seg. 4 

Seg. 5 

Seg. 6 

Seg. 7 

Average 
Time (sec) 

135.6 

6.8 

27.4 

4.8 

56.3 

5.4 

64.0 


Approach Measures 

For data analyses, all approach runs were separated into seven segments to reflect different tasks the 
subject pilots were attempting to perform during the course of the approach maneuvers: straight-and- level 
(constant heading); initiate turn (localizer intercept); constant turn; roll out; localizer tracking; glideslope 
intercept (initiate descent); localizer and glideslope tracking (figure 12). In addition to analysis of the 
entire length of the run as a whole, these seven segments were also prioritized for data reduction and 
analyses. 



Figure 12. Top view of the approach maneuver segments 


The average run time for the approach maneuver was 6.25 min. On these approaches, all data runs 
were stopped at 200 ft AGL. The segment averages specific to the approach maneuver are detailed in 
table 7. 


24 


Table 7. Segment duration averages for approach maneuver 



Seg. 1 

Seg. 2 

Seg. 3 

Seg. 4 

Seg. 5 

Seg. 6 

Seg. 7 

Average 
Time (sec) 

63.4 

3.5 

33.4 

4.9 

116.5 

13.7 

140.3 


As previously mentioned, physiological data were collected during approaches in the form of the 
change in heart rate, skin temperature, and muscle response. ECG was recorded to determine if the 
change in subject peak-to-peak heart rate (difference in time between each corresponding R-wave peak) 
was due to the activities involved in the experiment. The R-wave is the point at which the ventricles 
contract to force blood to the body’s extremities from the heart muscle. This ventricular contraction 
produces the highest peak in the electrical signal that is shown in the data. SKT was monitored to 
ascertain if the test subject peripheral skin temperature changed relative to the experimental activities. 
Typically, the peripheral skin temperature decreases in the situation of “fight or flight” to try and protect 
the major organs of the body. As a result, whenever a stressful situation occurs, a decrease in skin 
temperature will ensue. To determine if sudden muscle electrical activity occurred during the run, EMG 
was observed. Whenever a particular muscle bundle is utilized, an electrical response occurs that is seen 
in the data as an electrical spike. The resting physiological conditions were recorded prior to each run. 
The delta values were calculated by measuring the difference between the resting state and the IMG 
physiological condition values. 

Rare-Event Measures 

The actual recorded time that the subject mentioned that something was amiss (i.e., the terrain looks 
too high) during the rare-event was the analyzed measure for this portion of the experiment. The results 
were divided into four categories. Category A indicates that the subject pilot was very aware of his/her 
surroundings, and indicated well in advance that they felt there was something amiss. The subject pilots 
in Category A were in a safe position and had plenty of time to maneuver to steer clear of terrain. Subject 
pilots who identified that something was amiss, but did so either within 500 ft or 5 seconds of impact 
were placed in Category B. Category B designated CFIT “incidents” - not necessarily a crash, but 
definitely a safety of flight issue. Category C contained subject pilots who identified that something was 
amiss, but were cued by OTW information, first. And, finally, Category D represented subject pilots who 
actually flew into terrain. 

Results and Discussion 

Learning and Fatigue Effects 

The run order for each subject pilot was randomized across all subject pilots, in order to attempt to 
minimize learning and fatigue effects on each subject. Table 8 shows the occurrence of each DC in run 
number one. 
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Table 8. Display concept occurrence for run number 1 (first en route run) 


Display Concept 

Occurrences 

Percentage 




CCFN1 

3 

11.1 

CCFN30 

0 

0.0 

EBG1 

4 

14.8 

EBGFN1 

2 

7.4 

EBGFN3 

1 

3.7 

EBGFN30 

1 

3.7 

PR1 

7 

25.9 

PRFN1 

1 

3.7 

PRFN3 

4 

14.8 

PRFN30 

0 

0.0 

BRD 

3 

11.1 

BSBG 

1 

3.7 


An SPSS (Statistical Package for the Social Sciences) analysis comparing the time within Level 1 
performance during the first en route run to all other en route runs showed a statistically significant result 
(p=.02). A subsequent post-hoc analysis showed that the statistical break was between run number one 
and run number two (see table 9, where N is the number of occurrences of that particular run). 

Table 9. Percent time within Level 1 Performance with respect to run numbers (all en route) 


runnum 

N 

Subset 

1 

2 

i 

26 

85.0638 


2 

27 


90.1830 

5 

27 


91.9637 

3 

27 


92.0700 

12 

27 


92.4633 

4 

26 


93.0127 

21 

27 


93.5952 

7 

27 


93.7730 

13 

27 


93.9456 

8 

27 


94.3752 

10 

27 


94.9730 

20 

27 


95.0207 

19 

26 


95.1338 

18 

27 


95.2452 

14 

27 


95.5407 

11 

26 


95.8715 

16 

27 


95.9385 

17 

27 


95.9537 

6 

27 


96.0007 

15 

27 


96.5152 

9 

26 


96.6762 

22 

26 


97.7677 

Sig. 


1.000 

.281 
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Since the occurrence of DCs for run number one was not equally distributed (i.e., some DCs appeared 
multiple times during the first run, where others didn’t occur at all), and because the statistically lower 
Level 1 performance for run number one suggests that the subject pilots were still acclimating to the 
simulator and learning the maneuver, all run number one’s were omitted from the overall data analysis. 

A similar analysis was performed on the approach runs. Table 10 shows the amount of occurrences 
of each DC for the first run of the approach series. 

Table 10. Display concept occurrence for run number 23 (first approach run) 


Display Concept 

Occurrences 

Percentage 




CCFN1 

2 

7.4 

CCFN30 

2 

7.4 

EBG1 

2 

7.4 

EBGFN1 

1 

3.7 

EBGFN3 

5 

18.5 

EBGFN30 

2 

7.4 

PR1 

4 

14.8 

PRFN1 

3 

11.1 

PRFN3 

2 

7.4 

PRFN30 

3 

11.1 

BRD 

1 

3.7 

BSBG 

0 

0.0 


An SPSS analysis comparing the time within Level 1 performance during the first approach run to all 
other approach runs showed a statistically significant result (p=.01). A subsequent post-hoc analysis 
revealed that the statistical break is not as evident in this case (table 11), indicating that the learning 
effect, while evident, is not as pronounced as it was for the en route case. Since the learning effect was 
not as evident in the approach runs, the first approach run was not omitted from the overall data analysis. 
As a check, the Level 1 performance analysis for the approach maneuver was performed with and without 
the first approach run, and the change in statistical power and significance was miniscule. 


Table 11. Percent Time within Level 1 Performance with respect to run numbers (approach) 


run n um 

N 

Subset 

1 

2 

23 

23 

84.0722 


26 

24 

84.2638 


27 

27 

89.6496 

89.6496 

24 

22 

92.3891 

92.3891 

25 

25 

92.7960 

92.7960 

28 

23 

92.8157 

92.8157 

31 

23 

93.0448 

93.0448 

34 

20 

93.3285 

93.3285 

33 

24 

93.6433 

93.6433 

29 

25 

93.9212 

93.9212 

32 

24 

94.0292 

94.0292 

30 

22 


95.6823 

Sig. 


.146 

.780 
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Qualitative Data 


SART Results 

One of the subjective ratings that will be discussed in terms of display type (versus DC) is the SART. 
The SART measure was calculated based on a combination of individual ratings of certain characteristics, 
such as understanding of the situation and demand on attentional resources. Figure 13 illustrates the 
SART rating data for each maneuver and how the different pilot groups rated each display type. 
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Figure 13. Mean SART results for each maneuver 
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For the high-altitude maneuver, post-hoc analyses revealed a statistically significant difference in the 
lower levels of SA that the BRD and the basic PFD provided to the higher levels of SA that resulted from 
adding terrain to the PFD. The biggest improvement of SA was realized when the terrain was added to 
the PFD, as compared to integration of pilot information on the PFD. For the low-altitude maneuver, 
subsequent post-hoc analyses grouped the three display types in independent sub-groups, meaning that 
the SA provided by each display type was statistically significant from each other, with the BRD 
providing the least SA, and the PFD with SVS terrain providing the most SA. The increase in SA from 
the BRD to the PFD, and from the PFD to the SVS-PFD, appear to be about equal. The difference 
between the results from the two maneuvers may be explained by the fact the altitudes for the two 
maneuvers are different. Pilots commented that at the lower altitudes, SA with regard to terrain presented 
on the display is much more important than at the higher altitudes. Therefore, the conclusion may be 
drawn that the nuances between the displays become more important at lower altitudes. Additionally, 
pilot opinion varied when comparing the BRD to the PFD and the SVS-PFD, with the more experienced 
pilots appearing to be more comfortable with the level of SA that they felt traditional gauges provided. 

For the approach maneuver, the low-time VFR- and IFR-rated pilots indicated that the biggest 
improvement in situational awareness occurred between the BRD and PFD, mostly reflecting the benefits 
of the tunnel in the sky guidance (rather than information integration), with a similar level of 
improvement with the addition of terrain on the SVS-PFD. As with the en route maneuvers, pilot opinion 
varied when comparing the BRD to the PFD, with more experienced pilots appearing to be more 
comfortable with the traditional gauges than the less experienced pilots. Most of the subject pilots 
commented that the velocity vector/guidance tunnel combination greatly enhanced their ability to fly an 
ILS approach. All pilots remarked that the presence of terrain on the F1DD enhanced their overall SA, 
increasing pilot SA almost as much as the improvement observed for the PFD with FIITS over the BRD. 
This is a powerful result, indicating the level of SA improvement due to the presence of terrain on the 
PFD is approximately equal to the dramatic improvement created by a PFD with guidance tunnel. 

Overall, the data show a general trend, regardless of maneuver, of improved SA from the traditional 
gauges to the integrated PFD, and then another improvement in SA with the addition of terrain. This 
trend agrees with pilot comments that the addition of terrain on the PFD provides a substantial increase of 
SA over the traditional gauges. 

TLX (Workload) Results 

Another subjective measure that is discussed in terms of display type is the NASA TLX rating. The 
TLX rating estimated workload through a combination of the individual ratings of mental demand, 
physical demand, temporal demand, pilot performance, effort, and frustration. Overall, a reduction in 
TLX was realized when the pilots were flying with the SVS displays. Figure 14 shows the TLX results 
for each of the maneuvers. 
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Figure 14. Total TLX (total workload) for each maneuver 

For the en route maneuvers, the data suggest that a small (but not statistically significant) decrease in 
perceived workload occurred between the BRD and the PFD, with a larger decrease between the PFD and 
the SVS-PFD. For the approach maneuver, the TLX data indicates that the low-time VFR- and IFR-rated 
pilots noted a substantial perceived workload decrease between the BRD and PFD, as well as a smaller 
(but still sizeable) decrease from the PFD to the SVS PFD (figure 14c). These TLX data complement the 
SART data, suggesting that when the terrain portrayal is added to the PFD, all pilots gained improved SA 
and an associated decrease in perceived workload, when compared to flying with the BRD. 
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Terrain Awareness Results 


Terrain awareness (TA) was assessed by subject pilots subjectively rating the level of TA each concept 
provided, using a scale of low TA (0) to high TA (10). Figure 15 illustrates the level of increased TA for 
all DCs for all maneuvers. 




a) High Altitude b) Low Altitude 



c) Approach 


Figure 15. Mean Terrain Awareness results for each maneuver 

For the en route maneuvers, statistical analyses indicate that the effect of DC was highly significant for 
all maneuvers. Subsequent Post-Hoc analysis for the high-altitude maneuver data indicated that the two 
baselines had statistically lower TA levels than all of the SVS display concepts, even though the baselines 
were evaluated in the presence of the MX-20. Additionally, the CCFN30 and the CCFN1 DCs yielded 


31 


higher values of TA than the two baselines, but were still statistically lower than all other SVS DCs. 
Post-Hoc analysis for the low-altitude maneuver data revealed that the subtle nuances between each DC 
were more apparent at lower altitudes in a terrain challenged environment (i.e., over Poor Mountain) with 
a potentially higher workload. The SNK post-hoc revealed nine subsets for the 12 DCs. The general 
trends are the same as for the high-altitude maneuver data (baselines yielding the lowest TA, with the 
more complex, higher-resolution DCs yielding higher TA), but in the low-altitude case, the EBGFN1 DC 
statistically stands out as having the best TA associated with it. 

For the approach maneuver, subsequent post-hoc analysis revealed again that the two baselines had 
lower TA levels and were statistically significantly different than all of the SVS display concepts, even 
though they, too, were evaluated in the presence of the MX-20. Additionally, while the CCFN1 and the 
CCFN30 DCs produced higher values of terrain awareness than the baselines, these display concepts were 
still significantly different than the rest of the SVS DCs. The post-hoc analysis also provided data that 
showed that while all SVS DCs supplied increased terrain awareness, they don’t convey similar amounts 
of terrain awareness information. The PRFN30 DC yielded a higher terrain awareness level than the 
CCFNs, but significantly lower level than the two highest-resolution EBGs. While it appears that all the 
SVS DCs are viable, texture and resolution combinations are important. Results from this statistical 
analysis indicate that for the EBG texture, DEM resolution doesn’t matter, whereas for the PR texture, 
DEM resolution is a contributing factor TA (hence the lower rating of the PRFN30 concept). Only at the 
lower-resolution DEMs do the differences between EBG and PR become evident. 

Overall, the results for the TA evaluations agreed with pilot comments that for the low-resolution 
DEMs, the coloring of the EBG was very intuitive and provided more information than the predominantly 
green color (due to vegetation) of the PR textures. Subject pilots also indicated that, in general, the EBG- 
textured DCs provided terrain information that was more intuitive and easier to interpret, agreeing with 
information contained in reference [11]. Specific pilot comments reflected a desire to know when they 
were approaching a ground-based hazard without a need to know whether it was rocks, dirt, or trees. 
Additionally, pilots indicated that while the 3 arc-sec DEM will be more than sufficient for GA 
applications, the 1 arc-sec DEM was still the preferred database resolution. The subject pilots also 
generally agreed that while the PR texture provided a “pretty picture”, the EBG texture was easier to 
decipher in terms of terrain undulations. 

Stress Results 

Perceived stress was assessed by subject pilots subjectively rating the level of stress they felt while 
flying each display type provided, using a scale of low stress (0) to high stress (10). Figure 16 illustrates 
the level of stress evaluated for display types for each maneuver. 
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Figure 16. Mean Stress results for each maneuver 

For the high-altitude en route maneuver, subsequent Post-Hoc analyses revealed that the subject pilots 
reported stress being significantly lower when flying the SVS-PFD than when flying the BRD. 
Additionally, for the low-altitude en route and the approach maneuvers, the stress perceptible while flying 
the BRD was significantly more than when flying the PFD, which in turn was significantly more than 
flying the SVS-PFD. The data indicate that when flying at lower altitudes, subjects felt that the stress 
level would significantly decrease when using the integrated PFD with the addition of terrain. 
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Display Rankings 


During the exit interview, subject pilots were asked to rank-order their preference for each texture on a 
scale of 1 to 6 (with 1 being the most preferred, 6 being the least preferred), and then for each DEM on a 
scale of 1 to 3, regardless of maneuver, with the baselines combined as one display concept. The 
following tables show the actual, discrete rankings for each pilot with respect to texture (table 12) and 
DEM (table 13), as well as the averages of these rankings for each terrain portrayal element. 


Table 12. Subject Pilot texture rankings 


Pilot 

EBG 

EBG+FN 

PR 

PR+FN 

CC+FN 

Baseline 

1 

2 

1 

4 

3 

5 

6 

2 

2 

4 

1 

3 

5 

6 

3 

1 

2 

3 

4 

5 

6 

4 

1 

2 

3 

4 

5 

6 

5 

1 

2 

3 

4 

5 

6 

6 

1 

3 

2 

4 

5 

6 

7 

2 

1 

4 

3 

5 

6 

8 

1 

2 

4 

3 

5 

6 

9 

2 

4 

1 

3 

5 

6 

10 

2 

1 

4 

3 

5 

6 

11 

5 

3 

2 

1 

4 

6 

12 

3 

1 

4 

2 

5 

6 

13 

4 

2 

3 

1 

5 

6 

14 

2 

1 

4 

3 

5 

6 

15 

1 

2 

3 

4 

5 

6 

16 

3 

4 

1 

2 

5 

6 

17 

3 

4 

1 

2 

5 

6 

18 

2 

4 

1 

3 

5 

6 

19 

3 

4 

1 

2 

5 

6 

20 

2 

1 

4 

3 

5 

6 

21 

4 

3 

2 

1 

5 

6 

22 

4 

3 

2 

1 

5 

6 

23 

3 

4 

1 

2 

5 

6 

24 

3 

4 

1 

2 

5 

6 

25 

2 

4 

1 

3 

5 

6 

26 

1 

3 

2 

4 

5 

6 

27 

3 

4 

1 

2 

5 

6 

Max 

5 

4 

4 

4 

5 

6 

Min 

1 

1 

1 

1 

4 

6 

Avg 

2.3 

2.7 

2.3 

2.7 

5.0 

6.0 

St Dev 

1.1 

1.2 

1.2 

1.0 

0.2 

0.0 
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Table 13. Subject pilot DEM rankings 



PR Texture 


EBG Texture 


CCFN Texture 

Pilot 

PRFN30 

PRFN3 

PRFN1 


EBGFN30 

EBGFN3 

EBGFN1 


CCFN30 

CCFN1 

1 

3 

2 

1 


3 

2 

1 


2 

1 

2 

3 

2 

1 


3 

2 

1 


2 

1 

3 

3 

2 

1 


3 

2 

1 


2 

1 

4 

3 

2 

1 


3 

2 

1 


2 

1 

5 

3 

2 

1 


3 

2 

1 


2 

1 

6 

3 

2 

1 


3 

2 

1 


2 

1 

7 

3 

2 

1 


3 

2 

1 


2 

1 

8 

3 

2 

1 


3 

2 

1 


2 

1 

9 

3 

2 

1 


3 

2 

1 


2 

1 

10 

3 

2 

1 


3 

2 

1 


2 

1 

11 

3 

2 

1 


3 

2 

1 


2 

1 

12 

3 

1 

2 


3 

1 

2 


2 

1 

13 

3 

2 

1 


3 

2 

1 


2 

1 

14 

3 

2 

1 


3 

2 

1 


2 

1 

15 

3 

2 

1 


3 

2 

1 


2 

1 

16 

3 

2 

1 


3 

2 

1 


2 

1 

17 

3 

2 

1 


3 

2 

1 


1 

2 

18 

3 

2 

1 


3 

2 

1 


2 

1 

19 

3 

2 

1 


3 

2 

1 


2 

1 

20 

3 

2 

1 


3 

2 

1 


2 

1 

21 

3 

2 

1 


3 

2 

1 


2 

1 

22 

3 

2 

1 


3 

2 

1 


2 

1 

23 

3 

2 

1 


3 

2 

1 


2 

1 

24 

3 

2 

1 


3 

2 

1 


2 

1 

25 

3 

2 

1 


3 

2 

1 


2 

1 

26 

3 

2 

1 


3 

2 

1 


2 

1 

27 

3 

2 

1 


3 

2 

1 


2 

1 

Max 

3 

2 

2 


3 

2 

2 


2 

2 

Min 

3 

1 

1 


3 

1 

1 


1 

1 

Ave 

3.0 

2.0 

1.0 


3.0 

2.0 

1.0 


2.0 

1.0 

STDV 

0 

0.2 

0.2 


0 

0.2 

0.2 


0.2 

0.2 


The subject pilots consistently ranked the textures EBG and PR the same, in terms of preference, as 
well as always rating the baseline concepts the least preferred. Additionally, with the exception of two 
subject pilots, the CCFN texture was constantly ranked next to last. Nearly all of the pilots commented 
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that the CCFN textures provided the least amount of terrain information of those evaluated, but all pilots 
said that they would gladly take the CCFN texture over the traditional gauges. With respect to the DEMs, 
the higher-resolution DEMs were consistently ranked higher than the lower-resolution DEMs. 

Specific questions were asked with respect to the FN overlay, and pilot comments varied. The subject 
pilots typically either strongly felt the FN provided enhancements to the terrain definition (like ground 
following and depth cues), or they really felt that the FN detracted from the terrain portrayal (because of 
reasons such as the FN being easily mistaken for roads and/or rivers). There didn’t seem to be a middle 
ground. When asked what could be done to improve the FN, common answers were: make the grid some 
complementary color to the texture, and change colors when the texture does, so you don’t lose the FN in 
the texture; experiment with the grid spacing (for example, closer together to indicate steeper terrain); and 
make it pilot selectable. In general, subject pilots indicated that they would not “pay” much for the FN 
option. 

The qualitative preference for the different SVS display concepts was evaluated by rank-ordering each 
display concept (except the CCFN30, no tunnel case) for approach, en route, and hypothetical emergency 
situations. The BRD and BSBG were grouped together. Table 14 shows the averages of these rankings 
for the cases evaluated. 


Table 14, Average of display concept rankings 



Approach 

En Route (All) 

Emergency 

Rank 

Actual 

Mean 

Display 

Concept 

Actual 

Mean 

Display 

Concept 

Actual 

Mean 

Display 

Concept 

1 

2.67 

EBG1 

2.63 

PR1 

2.44 

PR1 

2 

2.70 

PR1 

2.67 

EBG1 

2.85 

PRFN1 

3 

3.15 

PRFN1 

3.04 

PRFN1 

2.89 

EBG1 

4 

3.30 

EBGFN1 

3.37 

EBGFN1 

3.48 

EBGFN1 

5 

5.04 

EBGFN3 

5.11 

EBGFN3 

4.93 

PRFN3 

6 

5.15 

PRFN3 

5.15 

PRFN3 

5.19 

EBGFN3 

7 

7.22 

EBGFN30 

7.11 

PRFN30 

7.07 

PRFN30 

8 

7.30 

PRFN30 

7.30 

EBGFN30 

7.33 

EBGFN30 

9 

8.59 

CCFN1 

8.74 

CCFN1 

8.93 

CCFN1 

10 

9.96 

CCFN30 

9.96 

CCFN30 

9.96 

CCFN30 

11 

10.93 

Baselines 

10.93 

Baselines 

10.93 

Baselines 


A non-parametric test was conducted on each set of data to determine statistical significance and 
selected case pairs were further investigated individually using a related samples procedure (Wilcoxon 
test). For all cases, subject pilots consistently ranked the two CCFN terrain portrayals and the baseline 
concepts as the least preferred display concepts. Another statistically-significant division occurred 
between the 1 and 3 arc-sec DEMs for the EBG and PR textures. The emergency scenario yielded a 
slightly different (but not statistically significant) result in that both of the PR textures with the highest- 
resolution DEM were consistently ranked higher than both of the EBG textures with the highest- 
resolution DEM. Pilots commented that during an emergency situation, they felt that it would be valuable 
to know which features make up the ground (trees, buildings, etc.), to aid in the decision-making process 


36 




of where to make an emergency landing. In conclusion, although minor preference variations occurred, 
statistically, the EBG1, PR1, EBGFN1, and PRFN1 terrain-texturing concepts are interchangeable; 
EBGFN3 and PRFN3 are comparable; and EBGFN30 and PRFN30 are also interchangeable. In addition, 
statistical results show that CCFN1 was consistently ranked higher than CCFN30, which in turn was 
consistently ranked higher than the baseline concepts. 

SA-SWORD Results 

During the block questionnaires, the subject pilots were asked to compare one terrain-texturing 
concept versus another, in a pair-wise comparison (across all terrain-portrayal concepts), using a nine- 
point scale. The results of these comparisons are reported in terms of a geomean, which returns the 
geometric mean of an array or range of positive data and is relevant any time several quantities multiply 
together to produce a product. The geomean is then translated into a rating to indicate the relative rank of 
each DC in terms of SA. This method of data collection is slightly different than the aforementioned 
rankings in that these are not discrete data, but a derivation of a ranking based on subject input producing 
relative scale values. The following table, table 15, shows the results of the geomean rankings, in order. 


Table 15. SA-SWORD Geomean results 



High Altitude 

Low Altitude 

Approach 

Avg. 

Ranking 

Geomean 

Display 

Concept 

Geomean 

Display 

Concept 

Geomean 

Display 

Concept 

1 

.193 

PR1 

.186 

PR1 

.176 

PR1 

2 

.174 

PRFN1 

.185 

PRFN1 

.175 

PRFN1 

3 

.170 

EBGFN1 

.168 

EBGFN1 

.163 

EBGFN1 

4 

.159 

EBG1 

.162 

EBG1 

.155 

EBG1 

5 

.091 

EBGFN3 

.091 

EBGFN3 

.092 

PRFN3 

6 

.084 

PRFN3 

.085 

PRFN3 

.086 

EBGFN3 

7 

.042 

EBGFN30 

.040 

EBGFN30 

.043 

PRFN30 

8 

.041 

PRFN30 

.038 

PRFN30 

.037 

EBGFN30 

9 

.027 

CCFN 1 

.022 

CCFN 1 

.022 

CCFN 1 

10 

.020 

CCFN30 

.022 

CCFN30 

.015 

CCFN30 


Across all maneuvers and all DC, the relative SA-SWORD results were very similar. The PR and 
EBG display concepts always rated higher than the CCFN concepts. The general trend was for the PR 
concepts to yield a higher rating than the EBG concepts, but not in terms of statistical significance. 
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ANOVAs performed on each SA-SWORD data set revealed similar results as before, regardless of 
maneuver. EBGFN1, EBG1, PRFN1, and PR1 were statistically interchangeable; PRFN3 and EBGFN3 
are comparable; and, in this case, the lower resolution DEMs of the PRFN and EBGFN texturing 
concepts were statistically equivalent to the two CCFN display concepts. 

Field of View 

In terms of FOV, the subject pilots were given the opportunity to select which two FOVs they would 
prefer to use, if they could only select two to use in their airplane. Table 16 shows which FOVs that each 
subject pilot selected. FOV of 60° was selected the most often, because it was a good balance between 
FOVs of 30° and 90°. The subject pilots commented that the FOV of 60° was the most usable and 
controllable (with the FOV of 90° having too much compression to the point of being potentially 
misleading, and FOV 30° being too visually active) for cruise and initial approach. But, once the pilot 
was on short final, they liked to switch to the narrower FOV to more closely match the OTW image, and 
for more precise velocity-vector sensitivity. 

Table 16. Top two FOV choices per pilot 



FOVs 




Pilot 

22° 

U> 

O 

o 

Os 

O 

o 

kO 

o 

o 

1 


✓ 



2 




✓ 

3 

V 



✓ 

4 



✓ 

✓ 

5 



✓ 

✓ 

6 



✓ 

✓ 

7 

V 


✓ 


8 


✓ 



9 

V 


✓ 


10 

V 


V 


11 

V 
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12 


✓ 
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13 
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14 
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15 
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16 



V 

✓ 

17 


✓ 


✓ 

18 
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19 
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✓ 

20 
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21 
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22 

V 


V 


23 


✓ 

V 


24 


✓ 

V 


25 


✓ 

V 


26 

V 


V 


27 


✓ 

V 


Totals 

13 

8 

25 

8 
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Tunnel-On Versus Tunnel-Off Qualitative Results 

In the case of the tunnel-on versus tunnel-off for the CCFN30 DC, as expected, the pilots’ SA 
decreased while workload and stress levels increased without the guidance tunnel present (see figure 17). 



No Tunnel Tunnel No Tunnel Tunnel 


a) Total TLX (workload) 


b) Mean SART (situation awareness) 



No Tunnel Tunnel 


c) Mean Stress 


Figure 17. Tunnel-off versus tunnel-on qualitative results for CCFN30 display concept 

Additionally, subject pilot comments strongly indicated that the use of tunnel guidance provided 
substantial information to facilitate flying the approach maneuver within the identified performance 
parameters and was much preferred over flying without a tunnel present on the display. When the 
subjects were asked to indicate the level of SA enhancement when using the tunnel during the approach 
versus no tunnel for the CCFN30 concept, the average response was that there was 85% more SA when 
using the tunnel. 
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Pilot Comments 


Pilot comments have been interleaved into both the quantitative and qualitative discussion section. 
However, a few themes evolved during the hundreds of hours of pilot comments that are worth 
emphasizing. 

Pilots did not seem to care if there were dirt or rocks or trees on the ground, as long as they understood 
what the underlying terrain was doing. This theme was derived from comments such as: 

• “For approach, bumps in ground are easier to pick up on slightly less detailed terrain (EBG).” 

• “Some of terrain detail can get lost as background colors (for the PR).” 

• “EBG - immediately obvious where low ground is, and that you should “fly to the green”.” 

• “(For the PRFN1) The terrain down there... there’s so many different colors it just kind of 
gets lost. The peaks, even the valleys, become... appears more flat compared to the elevation 
based (generic).” 

• “I don’t really need to see everything that PR has.” 

While CCFN, regardless of DEM, was the least-preferred terrain-texturing method, the pilots still 
emphatically preferred it over the traditional gauges. This notion emerged from comments such as: 

• “Not much good, but would take over standard gauges.” 

• “Beats hands down standard gauges.” 

• “CCFN - if had to use, could. It’s like bicycling across country versus riding in a Cadillac. But 
definitely would pick over traditional dials.” 

The information that is gained from the PR and the EBG terrain textures is similar enough that no 
definitive strong recommendation based on preference of one texture type over the other emerged. 
Although individual pilots may have had a strong preference of one over the other, a compilation of the 
pilot comments shows that ultimately the PR and EBG texturing concepts were a pretty close tie. This 
theme emerged from comments like: 

• “Information wise, EBG, EBGFN, PR, and PRFN are the same.” 
and an equal amount of comments such as: 

• “EBG by far the best.” 

• “PR is the most realistic:” 

While pilots said that they preferred the higher resolution DEMs over the lower resolution DEMs, they 
did say that the DEM of 3 arc-sec does a fairly good job at representing the terrain undulations, and that 
they would not buy the 1 arc-sec DEM if it cost any more than the 3 arc-sec DEM. This view transpired 
as a result of comments such as: 

• “Between DEMI and DEM3 is a tough call, but not sure how much better 1 is over 3.” 

• “I think 3 is good, but if cost wasn’t an issue, would choose DEMI.” 

• “The difference between 1 and 3 was appreciated, but not as critical.” 

• “DEM3 is pretty good, until you see DEMI. DEM3 would be adequate, if DEMI was too 
expensive.” 


40 



The tunnel guidance provides a tremendous amount of information and greatly facilitates pilot 
performance while reducing workload during ILS approaches. This opinion evolved from comments such 
as: 


• “Yeah this is definitely a lot harder to control. Workload is doubled it seems like. I think no 
tunnel in general makes a big difference.” 

• “The tunnel made a big difference. It took a whole lot more paying attention without the 
tunnel.” 

• “Wow, that’s certainly harder to do. . . Yeah, I really miss that tunnel. . . I’m trying to recapture 
here. Actually, you’re a lot busier now, and it’s harder even to pay too much attention to the 
terrain, but I do notice the obstacles.” 

• “I mean I was chasing the needle and the altitude just kind of going back and forth, it makes 
you really appreciate having that little tunnel.” 

In general, pilot comments correlated very well with the subjective and objective data. 

Quantitative Data 

En Route Maneuvers 

The high-altitude data were analyzed with the exclusion of the first en route run (run number one). 
Using the performance parameters discussed in the Quantitative Data Description section for the high- 
altitude, a significant effect was found for the Level 1 performance for the entire run (see figure 1 8), with 
the BRD cases yielding a much lower performance than the two integrated PFDs. 



BRD PFD SVS-PFD 

Display Type 


Figure 18. Level 1 Performance for the high-altitude maneuver 

A closer look at the data revealed that Segment 3 (constant turn and descent, using parameters of bank 
angle error < ± 10° and indicated airspeed (IAS) error < ± 10 knots) was the most problematic segment 
for the subject pilots in terms of Level 1 performance (see figure 19). 
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Figure 19. LI Performance and components for segment 3 (high-altitude) 

While the individual parameters were not found to have significance, the trends of these parameters 
were that the airspeed and bank angle were definitely worse while using the BRDs. The cumulative effect 
of these negative trends for these two parameters yielded a significant combined result. Additionally, the 
heading error for both segments 1 (straight and level, constant heading) and 5 (descent with constant 
heading) was found to be significant (see figure 20). For segment 1, the SNK post-hoc analysis revealed 
three different significant subgroups, with the PFD yielding the lowest heading error RMS and the BRD 
yielding the highest. For both segments, the heading error with the BRD cases was worse than that of the 
integrated PFDs, especially in segment 5. 
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Display Type 


a) Segment 1 Heading Error 



Display Type 


b) Segment 5 Heading Error 


Figure 20. Heading Error RMS (high-altitude) for segments 1 and 5 

Overall, these findings indicate that the tasks of negotiating the turns and maintaining 
headings with the much smaller attitude indicator on the BRD were more difficult for the subject 
pilots to accomplish than when using the integrated PFDs. These data also confirm that the 
addition of terrain with substantial increases in SA and reductions in workload will not degrade 
pilot performance. Pilot control activity was fairly consistent among the different display types 
for the high-altitude maneuver, suggesting that there wasn’t a considerable difference in the 
amount of physical workload (in this case measured by throttle, rudder, and pitch inputs) required 
to fly using each of these display types. 


For the low-altitude maneuver, the three display types performed equally well, in terms of 
pilot performance. Again, the addition of terrain did not impede performance. The pilot control 
activity indicates that in terms of physical workload, the subject pilots worked significantly 
harder with the rudder control (figure 21) for the BRD to maintain the same level of performance 
as they did with both the PFDs. However, the throttle and pitch recorded inputs were similar 
among the three display types. 



Display Type 


Figure 21. Rudder Control Input Standard Deviation for low-altitude maneuver 
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Approach Maneuver 

Figure 22 shows the mean percent of time the pilots flew within desired performance for the 
final approach task, with respect to display type. As anticipated, the less-experienced VFR pilots 
had difficulty flying a precision approach using the BRD, achieving the desired performance less 
than 40% of the time, demonstrating the fact that this category of pilots could not safely perform 
these types of pilot tasks with conventional instrumentation. Low-time IFR-rated pilots 
accomplished desired performance approximately 63% of the time, achieving minimal acceptable 
performance (as defined in this experiment) using the BRD. The high-time pilots maintained 
desired performance using the BRD nearly 80% of the time. An increase in pilot performance 
was achieved for all pilots when flying with the PFD and SVS-PFDs in the presence of a 
guidance tunnel, with minimal training. The data also show that when using either the PFD or the 
SVS-PFDs, the low-time VFR-rated pilots were able to fly as well as the high-time pilots did 
using the BRD. In addition, when terrain was added to the PFD the impact to performance level 
for all pilot groups was negligible, regardless of DEM and/or texture, although the previously 
discussed qualitative results indicated greatly enhanced SA and decreased workload with terrain 
present. 



Display Type 


Figure 22. Level 1 Performance for the approach run 

Detailed examination of the flight performance for all pilots (including low-time pilots flying 
with the BRD) shows segment 7 (Localizer and Glide Slope tracking, maintaining constant 
airspeed) to be the most problematic segment during the approach maneuver. All of the 
performance parameters (figure 23), glide slope error, localizer error, and IAS, that collectively 
make up the Level 1 performance for segment 7 are statistically significant. For all of these 
separate performance parameters in segment 7, performance while flying with the BRD was 
statistically lower than performance while flying either the PFD or the SVS-PFD. Operationally 
speaking, the results from the mean glide slope tracking error may be the most meaningful, with 
the most apparent difference being nearly a half-dot in mean performance improvement from the 
BRD to the integrated PFDs. 
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c) IAS Error 


Figure 23. Performance parameters for segment 7 

Figure 24 shows the cumulative histogram of the glide slope error data for all display types 
(Mean=.43, Std. Dev.=. 23). Approximately 96% of the time, the pilots were able to stay within 
the Level 1 performance with respect to glide slope (< 1 dot). In the case of the BRD, the low- 
time pilots mean glide slope error was greater than one dot, consistently. Figure 25 shows the 
cumulative histogram for the BRD case (Mean=.95, Std. Dev.=.6), where the pilots were able to 
stay within 1-dot error approximately 69% of the time, indicating that the glide slope tracking 
maneuver was more difficult using the BRD case than with the integrated PFDs. 
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Figure 24, Histogram for segment 7 Glide Slope Error, for all display types 



Segment 7 BRD Glide Slope Error, Dots 


Figure 25. Histogram for segment 7 Glide Slope Error, for the BRD 

Figure 26 shows the mean percent of time that the subject pilots flew within desired 
performance for the final approach task, as well as the detailed examination of the different 
elements of flight performance, this time with the exception of the cases where the low-hour VFR 
pilots were flying the BRD, because low-time VFR pilots flying an ILS with standard gauges is 
not a typical event in current operations. The overall mean for the BRD cases did improve (about 
14%) when the data from the low-hour VFR pilots were excluded, but not enough to make a 
statistically significant difference in the results when compared to the other two display types (the 
PFD and SVS-PFD), indicating that regardless of pilot type, integrated PFDs with HITS guidance 
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symbology does improve overall performance. Through comparison of the PFD and the BRD 
data (figure 26), addition of the HITS improved overall performance, extending the mean 
percentage of time spent within Level 1 from 68.6% (BRD, all pilots) to 91.9% (PFD, all pilots). 
Additionally, improved performance was maintained when terrain was added to the PFD, 
comparing the PFD data, all pilots, versus the SVS-PFD data, all pilots. As a reminder, in figure 
26, the “All Pilots” category does exclude the cases where the low-hour VFR pilots were flying 
the BRD. 
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Figure 26. LI Performance (entire run) and individual performance parameters for 
segment 7, with the low-hour VFR/BRD case excluded 

Figure 27 shows the cumulative histogram of the glide slope error data for all display types 
(Mean=.42, Std. Dev.=.25), but without the low-time VFR BRD case. As with the case where the 
low-time VFR BRD data were included, approximately 97% of the time, for all DCs, the pilots 
were able to stay within the Level 1 perfonnance with respect to glide slope. However, looking 
at the BRD display type specifically, figure 28 shows the cumulative histogram (Mean=.66, Std. 
Dev.=.32), where the pilots were able to stay within 1-dot error approximately 85% of the time 
(as opposed to 69% when the low-time VFR pilots are included), indicating that, regardless of 
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pilot experience, the glide slope tracking maneuver was more difficult using the BRD than with 
the integrated PFDs. 



Segment 7 Glide Slope Error, Dots 
No Low-Time VFR/BRD Case 


Figure 27. Histogram for segment 7 Glide Slope Error, for all display types, low-hour 

VFR/BRD case excluded 



Segment 7 Glide Slope Error, Dots 
No Low-Time VFR Pilots 


Figure 28. Histogram for segment 7 Glide Slope Error, for the BRD, low-hour VFR pilots 
excluded 

Analyses were conducted on pilot control inputs using the standard deviations of pilot pitch, 
roll and throttle control. The physical workload (as defined by the standard deviations in pilot 
control activity) variations were small between the BRD and the two PFDs, with no statistical 
significance. Therefore, pilots were able to realize a significant improvement in pilot 
performance without compromising the physical workload associated with flying. 

For the conditions reported in this paper, flight-path control was improved with the PFD and 
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SVS-PFD concepts. Although these data are not shown, different terrain presentations did not 
appear to impact flight performance in this study. The improvement in performance from the 
BRD to the PFD and SVS-PFD concepts was attributed to the advanced symbology (velocity 
vector, FIITS, etc.) associated with these advanced PFDs. 

Physiological Data Analysis Results 

The physiological data were analyzed to determine if the various display concepts produced 
measurable physiological differences in the subject pilots that would indicate a change in 
perceived workload. Analyses were performed across all subject pilot experience levels, and a 
general trend of the overall physiological responses of the subject pilots emerged, indicating that 
the subjects favored SVS displays over the BRD. For more information on the physiological 
results, see reference 16. 

Field of View (FOV) Results 

An analysis of the FOV selected by the subject pilots was performed to determine if FOV use 
was different for the various DCs since certain aspects of the symbology employed for this 
experiment present flight path errors differently for different FOVs. For the en route maneuvers, 
subject pilots could use the velocity vector to manage their rate of descent by placing the velocity 
vector on a specific location on the PFD for specific airspeeds. Changing FOVs would alter 
where specific flight path angles would be located on the display. For example, higher FOVs 
would move the -5° reference line closer to the center of the display as well as make the 
difference between -5° and -10° appear smaller. In addition, the subject pilots’ guidance for the 
approach task with the HITS was to place the velocity vector into the center of the tunnel boxes. 
Higher FOVs would change the appearance of a given amount of flight path error and make 
larger errors seem smaller. The converse is also true in that lower FOVs would make flight path 
errors seem larger. These changes in the appearance of flight path error could change the subject 
pilot’s control strategy and affect the resulting FTE. Based on figure 29 and the ANOVA, subject 
pilots’ selection of FOV was similar for all DCs, indicating that pilot performance was not 
impacted by the FOV selection. 
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a) High Altitude b) Low Altitude 



c) Approach 

Figure 29. Field of view selection across all display concepts 


Tunnel-On Versus Tunnel-Off Quantitative Results 

For the CCFN30 tunnel-off versus tunnel-on evaluation, as expected, the tunnel enhanced 
performance. The mean percent time within Level 1 performance for the tunnel-off case was 
62.5%, while the tunnel-on mean was 93.4%. For eight of the 27 subject pilots, flying with a 
guidance tunnel present on the display substantially improved their performance scores by at least 
a factor of 2 (table 17). Asa result, seven of the eight subject pilots transcended two performance 
levels, moving from a below-adequate performance rating into a desired performance rating, 
while one of the eight subject pilot’s performance rating improved from an adequate to a desired 
performance rating. Of these eight pilots, 50% of them were in the low-time VFR category, 25% 
were low-time IFR-rated, and 25% of them were high-time pilots. 
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Table 17. Summary of Tunnel-Off versus Tunnel-On cases for select pilots 


Subject 

Pilot 

Experience 

Category 

CCFN30-No 
Tunnel 
(% Time 
within LI) 

CCFN30-with 
Tunnel 
(% Time 
within LI) 

2 

VFR 

23 

92 

3 

VFR 

36 

75 

4 

IFR 

48 

100 

6 

High-Time 

27 

97 

9 

VFR 

37 

99 

11 

High-Time 

52 

96 

16 

IFR 

45 

94 

25 

VFR 

56 

97 


Quantitative, Rare-Event Maneuver 

The rare event maneuver was developed to determine if SVS terrain would provide the 
necessary cues to avoid a CFIT event. As in the en route scenarios, the subject pilots were cued 
to initiate the turn by a waypoint on the MX-20, introducing flight-path variability to the 
maneuver. If the pilot made the turn early or late, they would consequently fly over different 
areas. However, the natural layout of the ridge was such that regardless of where the pilot 
initiated the turn, the failed altimeter would result in a hazardous situation (either impacting the 
simulated terrain or passing very closely over terrain). The measure analyzed for the rare-event 
maneuver was based on the actual time at which the subject pilot first mentioned that something 
was amiss (i.e., the terrain looks too close) (see Appendix E). The rare-event results were 
grouped into four categories. Category A contains those subjects who were very aware of their 
surroundings, and indicated well in advance that they felt there was something amiss. The 
subject pilots in Category A were judged to be in a safe position and had adequate time to 
maneuver to steer clear of terrain. Subject pilots who identified that something was amiss, but 
did so either within 500 ft of impact were placed in Category B. Category B was designated 
CFIT “incidents” - not necessarily a crash, but definitely a safety-of-flight concern. Category C 
indicated subject pilots who identified that something was amiss, but were first cued by OTW 
information (visibility on the OTW display was one statute mile), rather than their instrument 
displays. And, finally, Category D represented subject pilots who actually flew into terrain. Due 
to the nature of the failure (i.e., wrong baro setting in the MX-20 and altimeter), subject pilots had 
no correct altitude indication for the baseline conditions (i.e., 100% Category C/D), therefore, the 
baseline conditions were not evaluated during the rare-event maneuver. Table 18 summarizes the 
results, grouped by pilot experience. 

Table 18. Summary of rare-event category results 


Category 

Low- 

Time 

VFR 

Low- 

Time 

IFR 

High- 

Time 

Total 

A 

9 

5 

4 

18 

B 

0 

1 

2 

3 

C 

3 

0 

1 

4 

D 

2 

0 

0 

2 
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Figure 30 shows a contour plot of the region of terrain where the rare-event run was designed 
to end, with green designating a 3500-ft elevation contour, and the blue a 3750-ft contour. 
Within this figure, the dots indicate the point at which each subject pilot indicated that something 
was amiss or impacted the terrain. The two dots in the group of mountains (upper right set of 
contours) are the subject pilots that actually impacted the terrain. 
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Figure 30. Contour plot of rare-event maneuver end area 

In summary, for 18 out of 27 runs (67%), subject pilots noted that something appeared to be 
wrong well within the safe zone. Subject pilots that fell within category B, while safety of flight 
was a concern, did avoid terrain. Therefore, combining categories A and B, 78% of the subject 
pilots avoided a CFIT using SVS PFDs. Twenty-two percent (combining Categories C and D) of 
the runs did result in hazardous conditions. These data suggest that training on the use of SVS 
PFDs for CFIT prevention is necessary. Also, these data may reflect that rare-event testing is 
challenging and some pilots may have observed something was wrong, but did not verbalize it, or 
delayed verbalizing it. Additionally, after the rare-event run, several pilots commented that they 
kn ew they were in a simulation, so a level of realism was missing. Even though the subject pilots 
were briefed in the beginning to speak up if they thought something did not appear to be correct, 
they still assumed that the researchers wanted the subject pilots to fly the maneuver and “stick to 
the numbers”, as opposed to noting that something appeared to be out of place. 
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Conclusions 


SVS displays present computer-generated three-dimensional imagery of the surrounding 
terrain to greatly enhance pilot's situation awareness (SA), reducing or eliminating controlled 
flight into terrain (CFIT), as well as low-visibility loss of control (LVLOC) accidents. A critical 
component of SVS displays is the appropriate presentation of terrain to the pilot. At the time of 
this study, the relationship between the complexity of the terrain presentation and resulting 
enhancements of pilot SA and pilot performance had been largely undefined. The current 
research effort conducted a comprehensive evaluation of the effects of two primary elements of 
SVS terrain portrayal: terrain texture and digital elevation model (DEM) resolution. 

While flight performance was not significantly affected by various terrain-portrayal concepts 
created through DEM or texture combinations, terrain information of any combination tested in 
this experiment on a primary flight display (PFD) proved to be valuable in terms of increased SA 
and terrain awareness (TA). Significant increases in SA can lead to a change in the type and 
frequency of aviation accidents. Pilots consistently ranked Elevation-Based Generic (EBG) and 
Photo-Realistic (PR) terrain-texturing concepts approximately equal and always higher than the 
Constant Color with the Fish Net (CCFN) (which was always rated better than the baseline 
displays). In terms of terrain awareness (TA), the baseline concepts always yielded the least TA 
values, with the two CCFN display concepts coming next, and the rest of the display concepts 
(DCs) yielding the highest TA values. For the high-altitude case, the PR and EBG DCs were 
statistically equivalent. However, when the subject pilots were flying closer to the terrain, as in 
the case of the low-altitude en route and the approach maneuvers, the higher-resolution EBG DCs 
provided more TA. Pilot comments indicated that the EBG was more easily deciphered in terms 
of terrain variations than the PR texture. Since PR concepts require specialized computer- 
graphics resources that exceed current certified flight computer platforms, EBG concepts are 
therefore recommended. In addition, regardless of maneuver, while higher DEM resolutions 
were preferred, pilot comments indicated that 3 arc-sec DEM data were nearly as good as the 1 
arc-sec DEM. As a result, high-resolution DEMs may not be required to obtain sufficient SA 
from SVS displays. Lastly, the secondary FN texturing concept received mixed ratings, with 
some subject pilots finding the information that the FN provided to be useful and others 
determining that the FN was a distraction. None of the subject pilots commented that they would 
pay much for the FN option. In general, the subject pilots agreed that the preferred field of view 
(FOV) setting was approximately 60°. In some circumstances (short final in calm conditions), a 
narrower FOV was typically preferred. When asked which two FOVs they would prefer in their 
aircraft, the majority of the subject pilots selected the FOVs of 60° and 22°. 

With respect to the display-type comparisons, the integrated PFDs always performed better in 
terms of workload, stress, and SA as compared to the baseline round-dials (BRDs). When terrain 
was added to the PFD to form the SVS displays, increases in SA and minor reductions in 
workload were evident in comparison to the combination of an integrated PFD with highway-in- 
the-sky (HITS) guidance and a moving-map display with TA. This result indicates that adding 
SVS terrain to current state-of-the-art PFDs generates significant benefits in terms of SA without 
compromising workload. Another positive finding was that the HITS improved overall 
performance (over the BRDs), extending the mean percentage of time spent within Level 1 from 
68.6% to 91.9% (all pilots). Additionally, improved performance was maintained when terrain 
was added to the PFD. 

The tunnel-on versus tunnel-off comparison with the CCFN30 DC indicated that, regardless of 
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pilot experience level, the tunnel provides the information necessary for the pilot to increase 
performance when flying the precision approach as compared to their performance when flying 
using only glide slope and localizer indicators and a pathway displayed on the navigational 
display (ND). Seven of the 27 pilots showed an improvement of at least two-fold (transcending 
performance levels) when flying with the tunnel as compared to flying without. This result has 
very strong implications as to how safety can be improved with the addition of HITS technology 
on a PFD. The subject pilots overwhelmingly agreed that the addition of the tunnel on the 
CCFN30 display concept made flying incredibly easier and commented that the HITS technology 
is something that they would like included in SVS display technology. 

Additionally, nearly 80% of the subject pilots avoided CFIT when flying with the SVS PFDs 
during the rare-event scenario, compared to what would most assuredly be near 0% when flying 
with the traditional gauges. Lives could be saved with the added level of SA afforded by 
displaying terrain on the PFDs as another piece of information for the pilot to incorporate into the 
decision-making process. 

In conclusion, with minimal training, integrated PFDs with tunnel guidance improved 
performance over that exhibited using the traditional displays for the low-time VFR- and IFR- 
rated pilots. Furthermore, this experiment demonstrated that when terrain is added to the 
integrated PFD, SA for the pilot was drastically enhanced and mental workload decreased, with 
no degradation in pilot performance. The benefits of the PFD with tunnel guidance could be 
described as a major contribution to pilot performance developed during the past several decades. 
Addition of SVS terrain provides a comparable increase in performance and augments HITS 
symbology. Due to the improvement of pilot performance with integrated displays with advanced 
guidance symbology, along with the increased SA inherent in SVS displays, pilots will have 
enhanced capabilities to avoid accidents during periods of reduced visibility. 

In this study, increases in SA were documented. Future work should focus on further 
definition of the benefits of increased SA due to SVS terrain. Further definition of the relevance 
of these increases in SA and reductions in workload needs to be performed in order to better 
estimate potential increases in safety and operational benefits. It is assumed that meaningful 
operational benefits would greatly accelerate incoiporation of SVS into the GA fleet. In addition, 
future work should include a detailed fault-hazard analysis that focuses on failure modes and how 
SVS displays could be affected. 
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Appendix A: Run Lists 

The following tables (tables Al, A2, A3, and A4) present the run list for each subject. 


Table Al. Run list for subjects 1 through 7 


i : 

| Subject 1 | Subject 2 

| Subject 3 

| Subject 4 

| Subject 5 | Subject 6 

| Subject 7 ( 

| Block 1: High Altitude j 

Runt 

PRFN3 

EBG1 

EBGFN1 

CCFN1 

PRFN3 

CCFN1 

PRl 

Run2 

CCFN1 

PRFN3 

CCFN1 

EBGFN3 

EBGFN30 

EBGFN30 

EBGFN30 

Run3 

BSBG BL 

EBGFN1 

PRFN30 

PRFN1 

PRFN30 

CCFN30 

EBGFN1 

Run4 

EBG1 

BSBG BL 

EBGFN3 

EBGFN30 

EBGFN1 

PRFN3 

PRFN3 

Run5 

EBGFN3 

CCFN1 

EBG1 

BRD BL 

PRFN1 

EBGFN3 

PRFN30 

Run6 

PR1 

PRFN30 

PRFN3 

CCFN30 

BSBG BL 

EBGFN1 

BSBG BL 

Run7 

PRFN30 

EBGFN30 

EBGFN30 

PRFN30 

CCFN1 

PR1 

PRFN1 

Run8 

EBGFN30 

EBGFN3 

PRFN1 

PR1 

EBG1 

EBG1 

CCFN1 

Run9 

PRFN1 

PR1 

PR1 

PRFN3 

EBGFN3 

PRFN30 

CCFN30 

Run 10 

EBGFN1 

PRFN1 

CCFN30 

EBGFN1 

CCFN30 

BRD BL 

EBG1 

Run 11 

CCFN30 

CCFN30 

BSBG BL 

EBG1 

PR1 

PRFN1 

EBGFN3 

| Block 2: Low Altitude | 

Run 12 

BSBG BL 

PRFN1 

EBGFN3 

CCFN30 

PR1 

EBGFN30 

EBGFN1 

Run 13 

EBGFN3 

PRFN3 

CCFN1 

EBGFN30 

EBG1 

EBG1 

PRFN30 

Run 14 

PRFN30 

EBGFN30 

PRFN30 

PRFN1 

CCFN30 

PRFN30 

PRFN1 

Run 15 

CCFN1 

EBG1 

BSBG BL 

PRFN1 

PRFN3 

PRFN1 

CCFN1 

Run 16 

PR1 

PR1 

PRFN1 

CCFN1 

PRFN1 

CCFN1 

BSBG BL 

Run 17 

EBGFN1 

EBGFN1 

PR1 

EBGFN3 

BSBG BL 

PR1 

EBGFN3 

Run 18 

PRFN3 

BSBG BL 

PRFN3 

PR1 

CCFN1 

PRFN3 

CCFN30 

Run 19 

CCFN30 

CCFN1 

EBGFN1 

BRD BL 

EBGFN3 

CCFN30 

PRFN3 

Run20 

EBGFN30 

EBGFN3 

EBGFN30 

EBGFN1 

EBGFN30 

BRD BL 

EBGFN30 

Run2 1 

PRFN1 

PRFN30 

CCFN30 

PRFN30 

EBGFN1 

EBGFN3 

EBG1 

Run22 

EBG1 

CCFN30 

EBG1 

PRFN3 

PRFN30 

EBGFN1 

PRl 

| Block 3: Approach j 

Run23 

PRFN1 

PRFN1 

PRFN3 

CCFN1 

PR1 

PR1 

PRl 

Run24 

CCFN30 

CCFN30nt 

CCFN30nt 

PRFN3 

CCFN1 

PRFN1 

CCFN30 

Run25 

EBGFN3 

PRFN30 

CCFN30 

PRFN30 

BSBG BL 

EBGFN1 

CCFN30nt 

Run26 

BSBG BL 

EBGFN3 

EBGFN1 

CCFN30 

CCFN30nt 

CCFN1 

EBG1 

Run27 

PR1 

CCFN30 

PRFN1 

EBGFN3 

PRFN30 

EBGFN3 

PRFN1 

Run28 

EBG1 

EBGFN1 

BSBG BL 

EBGFN1 

CCFN30 

EBG1 

PRFN30 

Run29 

EBGFN30 

CCFN1 

EBGFN3 

CCFN30nt 

EBGFN30 

BRD BL 

PRFN3 

Run30 

CCFN30nt 

PR1 

PR1 

EBG1 

EBG1 

PRFN30 

EBGFN30 

Run3 1 

EBGFN1 

BSBG BL 

EBGFN30 

PRFN1 

EBGFN3 

CCFN30 

EBGFN1 

Run32 

CCFN1 

PRFN3 

CCFN1 

EBGFN30 

PRFN3 

CCFN30nt 

CCFN1 

Run33 

PRFN30 

EBGFN30 

EBG1 

BRD BL 

EBGFN1 

PRFN3 

BSBG BL 

Run34 

PRFN3 

EBG1 

PRFN30 

PR1 

PRFN1 

EBGFN30 

EBGFN3 

| Rare Event I 

| Run35 

| CCFN1 | PRFN3 

| EBGFN30 

| EBG1 

| CCFN30 | PRFN1 

| EBGFN3 j 
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Table A2. Run list for subjects 8 through 14 


| Subject 8 | Subject 9 | Subject 10 | Subject 11 | Subject 12 | Subject 13 | Subject 14 

Block 1: High Altitude 

Runl 

PR1 

PR1 

BRD BL 

EBGFN1 

EBG1 

PRFN3 

PR1 

Run2 

CCFN1 

EBGFN1 

EBGFN3 

PR1 

CCFN30 

EBGFN1 

BRD BL 

Run3 

EBGFN30 

PRFN3 

PR1 

EBG1 

EBGFN30 

EBGFN30 

PRFN30 

Run4 

CCFN30 

PRFN1 

PRFN30 

PRFN3 

PRFN1 

EBGFN3 

EBGFN3 

Run5 

BRD BL 

BSBG BL 

CCFN30 

EBGFN3 

EBGFN1 

PRFN1 

CCFN1 

Run6 

EBGFN1 

EBGFN3 

EBGFN1 

CCFN1 

PRFN3 

CCFN1 

CCFN30 

Run7 

EBGFN3 

PRFN30 

PRFN1 

BSBG BL 

BRD BL 

PRFN30 

EBGFN1 

Run8 

PRFN1 

EBG1 

EBGFN30 

EBGFN30 

CCFN1 

EBG1 

EBGFN30 

Run9 

PRFN30 

CCFN1 

EBG1 

PRFN1 

PR1 

BSBG BL 

EBG1 

Run 10 

EBG1 

CCFN30 

CCFN1 

PRFN30 

PRFN30 

CCFN30 

PRFN1 

Runl 1 

PRFN3 

EBGFN30 

PRFN3 

CCFN30 

EBGFN3 

PR1 

PRFN3 

| Block 2: Low Altitude j 

Run 12 

EBGFN3 

CCFN30 

BRD BL 

EBGFN30 

CCFN1 

PRFN1 

CCFN1 

Runl 3 

CCFN1 

EBG1 

PRFN30 

PRFN3 

EBGFN1 

BSBG BL 

EBG1 

Run 14 

PR1 

EBGFN30 

PR1 

EBGFN1 

BRD BL 

EBGFN1 

PRFN30 

Run 15 

CCFN30 

PR1 

PRFN1 

EBG1 

EBGFN30 

CCFN1 

EBGFN30 

Run 16 

PRFN1 

EBGFN1 

EBGFN3 

CCFN1 

EBGFN3 

EBG1 

EBGFN3 

Runl 7 

EBGFN30 

PRFN1 

CCFN30 

PRFN1 

CCFN30 

CCFN30 

EBGFN1 

Runl 8 

PRFN30 

CCFN1 

EBGFN1 

CCFN30 

PRFN30 

PRFN30 

CCFN30 

Run 19 

EBGFN1 

PRFN3 

EBG1 

PR1 

PRFN1 

EBGFN30 

PRFN3 

Run20 

EBG1 

PRFN30 

CCFN1 

EBGFN3 

PRFN3 

PRFN3 

PR1 

Run21 

BRD BL 

BSBG BL 

PRFN30 

PRFN30 

EBG1 

PR1 

PRFN1 

Run22 

PRFN3 

EBGFN3 

EBGFN30 

BSBG BL 

PR1 

EBGFN3 

BRD BL 

| Block 3: Approach \ 

Run23 

PRFN30 

EBGFN3 

CCFN30 

CCFN30 

EBGFN3 

EBGFN30 

EBG1 

Run24 

CCFN30 

CCFN1 

EBG1 

EBGFN30 

CCFN1 

CCFN30nt 

EBGFN30 

Run25 

PRFN1 

EBG1 

EBGFN1 

CCFN1 

CCFN30nt 

CCFN1 

PRFN1 

Run26 

EBGFN30 

PRFN30 

CCFN30nt 

PR1 

BRD BL 

CCFN30 

EBGFN3 

Run27 

BRD BL 

BSBG BL 

BRD BL 

EBG1 

EBGFN30 

BSBG BL 

PR1 

Run28 

EBGFN1 

EBGFN1 

PRFN3 

PRFN30 

PRFN1 

PR1 

PRFN3 

Run29 

EBGFN3 

PRFN1 

PR1 

BSBG BL 

EBGFN1 

EBGFN3 

CCFN1 

Run30 

EBG1 

PR1 

EBGFN30 

PRFN1 

EBG1 

EBG1 

PRFN30 

Run3 1 

PRFN3 

EBGFN30 

PRFN1 

EBGFN1 

PRFN30 

PRFN1 

EBGFN1 

Run32 

PR1 

CCFN30 

CCFN1 

PRFN3 

PR1 

PRFN3 

CCFN30 

Run33 

CCFN30nt 

PRFN3 

PRFN30 

EBGFN3 

CCFN30 

PRFN30 

CCFN30nt 

Run34 

CCFN1 

CCFN30nt 

EBGFN3 

CCFN30nt 

PRFN3 

EBGFN1 

BRD BL 

| Rare Event j 

| Run35 

| EBGFN1 

| PRFN30 | PR1 

| CCFN30 

| EBG1 | EBGFN1 

| PR1 j 
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Table A3. Run list for subjects 15 through 21 



| Subject 15 

| Subject 16 [ 

| Subject 17 

| Subject 18 

| Subject 19 

| Subject 20 

| Subject 21 I 

| Block 1: High Altitude j 

Runl 

EBG1 

BSBG BL 

CCFN1 

PRFN1 

BRD BL 

PR1 

EBGFN3 

Run2 

CCFN30 

CCFN30 

EBG1 

BRD BL 

EBGFN3 

PRFN30 

EBGFN1 

Run3 

EBGFN30 

PR1 

EBGFN30 

EBGFN30 

PRFN30 

CCFN1 

CCFN1 

Run4 

PRFN1 

EBGFN1 

EBGFN3 

CCFN30 

PRFN3 

EBGFN3 

CCFN30 

Run5 

PRFN3 

EBGFN3 

EBGFN1 

PRFN30 

CCFN1 

EBG1 

BRD BL 

Run6 

EBGFN1 

CCFN1 

PR1 

PRFN3 

PR1 

EBGFN1 

EBGFN30 

Run7 

BSBG BL 

PRFN3 

PRFN3 

CCFN1 

EBGFN1 

BRD BL 

PRFN3 

Run8 

EBGFN3 

EBG1 

PRFN30 

PR 

PRFN1 

EBGFN30 

PR1 

Run9 

PR1 

EBGFN30 

BSBG BL 

EBGFN1 

EBGFN30 

PRFN1 

PRFN1 

Run 10 

PRFN30 

PRFN1 

CCFN30 

EBG1 

CCFN30 

CCFN30 

EBG1 

Runl 1 

CCFN1 

PRFN30 

PRFN1 

EBGFN3 

EBG1 

PRFN3 

PRFN30 

| Block 2: Low Altitude ] 

Run 12 

EBGFN1 

BSBG BL 

PR1 

EBG1 

CCFN30 

BRD BL 

EBGFN3 

Run 13 

PR1 

CCFN1 

PRFN3 

EBGFN1 

EBGFN3 

EBGFN1 

PR1 

Run 14 

CCFN1 

CCFN30 

EBGFN1 

PRFN3 

PRFN1 

PRFN30 

EBG1 

Run 15 

EBGFN3 

EBG1 

CCFN30 

PRFN30 

EBGFN30 

CCFN1 

PRFN3 

Run 16 

CCFN30 

EBGFN30 

EBG1 

PR1 

PRFN3 

PRFN1 

EBGFN30 

Run 17 

PRFN1 

PRFN3 

CCFN1 

CCFN30 

CCFN1 

CCFN30 

CCFN30 

Run 18 

PRFN3 

PR1 

PRFN1 

BRD BL 

PR1 

PR1 

PRFN1 

Run 19 

PRFN30 

PRFN1 

BSBG BL 

EBGFN30 

BRD BL 

EBG1 

CCFN1 

Run20 

EBGFN30 

PRFN30 

PRFN30 

PRFN1 

EBG1 

PRFN3 

PRFN30 

Run2 1 

BSBG BL 

EBGFN3 

EBGFN30 

CCFN1 

PRFN30 

EBGFN30 

EBGFN1 

Run22 

EBG1 

EBGFN1 

EBGFN3 

EBGFN3 

EBGFN1 

EBGFN3 

BRD BL 

| Block 3: Approach i 

Run23 

CCFN1 

EBGFN30 

PRFN30 

BRD BL 

EBGFN1 

EBGFN30 

EBG1 

Run24 

EBGFN1 

EBGFN3 

CCFN1 

PRFN1 

EBG1 

PRFN30 

PRFN30 

Run25 

PRFN3 

CCFN30 

CCFN30 

PRFN30 

EBGFN3 

PRFN3 

PR1 

Run26 

EBG1 

PRFN30 

EBG1 

CCFN30 

BRD BL 

PR1 

BRD BL 

Run27 

PR1 

BSBG BL 

EBGFN30 

EBGFN1 

PRFN3 

BRD BL 

CCFN1 

Run28 

EBGFN3 

PRFN1 

EBGFN3 

EBG1 

CCFN1 

CCFN1 

CCFN30nt 

Run29 

PRFN30 

PR1 

PRFN1 

EBGFN3 

CCFN30nt 

EBGFN3 

EBGFN30 

Run30 

PRFN1 

EBG1 

CCFN30nt 

CCFN1 

PRFN1 

PRFN1 

PRFN3 

Run3 1 

EBGFN30 

CCFN30nt 

EBGFN1 

CCFN30nt 

EBGFN30 

EBG1 

EBGFN1 

Run32 

CCFN30 

EBGFN1 

PRFN3 

PRFN3 

CCFN30 

EBGFN1 

PRFN1 

Run33 

BSBG BL 

PRFN3 

BSBG BL 

EBGFN30 

PR1 

CCFN30 

CCFN30 

Run34 

CCFN30nt 

CCFN1 

PR1 

PR1 

PRFN30 

CCFN30nt 

EBGFN3 

| Rare Event 1 

| Run35 [ 

| PRFN1 

| EBGFN3 | 

| PRFN3 

| EBGFN3 

| PRFN30 

| PRFN30 

| CCFN30 i 
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Table A4. Run list for subjects 22 through 27 


1 ! 

| Subject 22 

| Subject 23 

| Subject 24 

| Subject 25 | 

| Subject 26 | 

| Subject 27 1 

| Block 1: High Altitude | 

Runl 

EBGFN30 

PRFN3 

EBG1 

PRl 

PRl 

BRD BL 

Run2 

CCFN1 

PR1 

CCFN30 

BRD BL 

CCFN1 

EBGFN30 

Run3 

PRFN30 

EBGFN3 

EBGFN1 

EBGFN3 

EBGFN1 

EBGFN3 

Run4 

EBG1 

CCFN30 

BRD BL 

EBG1 

PRFN3 

EBGFN1 

Run5 

PR1 

PRFN30 

CCFN1 

PRFN30 

PRFN30 

PRl 

Run6 

CCFN30 

BSBG BL 

EBGFN3 

PRFN3 

CCFN30 

PRFN3 

Run7 

EBGFN1 

PRFN1 

PRFN3 

EBGFN30 

EBGFN3 

CCFN30 

Run8 

EBGFN3 

EBG1 

PRFN30 

CCFN1 

EBGFN30 

EBG1 

Run9 

PRFN3 

EBGFN30 

PR1 

PRFN1 

EBG1 

CCFN1 

Run 10 

BRD BL 

EBGFN1 

EBGFN30 

EBGFN1 

BSBG BL 

PRFN30 

Runl 1 

PRFN1 

CCFN1 

PRFN1 

CCFN30 

PRFN1 

PRFN1 

| Block 2: Low Altitude | 

Run 12 

BRD BE 

CCFN1 

PRFN3 

EBGFN30 

CCFN1 

EBGFN30 

Run 13 

EBG1 

PRFN30 

PR1 

EBGFN1 

PRFN30 

CCFN30 

Run 14 

PRFN30 

CCFN30 

EBGFN3 

CCFN1 

EBGFN1 

EBG1 

Run 15 

CCFN30 

BSBG BL 

CCFN30 

EBGFN3 

PRFN1 

EBGFN3 

Run 16 

PR1 

PR1 

EBG1 

BRD BL 

PRl 

BRD BL 

Run 17 

EBGFN30 

EBG1 

BRD BL 

PRFN30 

PRFN3 

PRFN3 

Run 18 

PRFN3 

EBGFN30 

CCFN1 

PRl 

CCFN30 

PRFN30 

Run 19 

PRFN1 

EBGFN1 

EBGFN30 

PRFN3 

EBGFN30 

CCFN1 

Run20 

CCFN1 

PRFN1 

EBGFN1 

CCFN30 

EBGFN3 

PRl 

Run2 1 

EBGFN1 

PRFN3 

PRFN1 

PRFN1 

BSBG BL 

PRFN1 

Run22 

EBGFN3 

EBGFN3 

PRFN30 

EBG1 

EBG1 

EBGFN1 
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Run23 

PRFN3 

PRFN1 

EBGFN3 

PRFN30 

EBGFN3 

PRl 

Run24 

CCFN30nt 

EBGFN3 

EBG1 

EBGFN1 

EBGFN1 

EBGFN3 

Run25 

CCFN30 

PRFN3 

EBGFN1 

EBGFN3 

CCFN1 

EBGFN30 

Run26 

PR1 

CCFN1 

PRFN1 

BRD BL 

CCFN30 

CCFN30 

Run27 

EBG1 

PRFN30 

PRFN3 

PRFN3 

PRl 

PRFN30 

Run28 

CCFN1 

PR1 

BRD BL 

CCFN30nt 

EBG1 

CCFN30nt 

Run29 

PRFN30 

EBGFN1 

PRFN30 

PRFN1 

PRFN1 

CCFN1 

Run30 

EBGFN30 

BSBG BL 

CCFN1 

PRl 

PRFN3 

EBG1 

Run3 1 

EBGFN3 

EBGFN30 

CCFN30nt 

EBGFN30 

PRFN30 

BRD BL 

Run32 

PRFN1 

CCFN30nt 

EBGFN30 

EBG1 

BSBG BL 

PRFN3 

Run33 

EBGFN1 

EBG1 

PR1 

CCFN30 

CCFN30nt 

PRFN1 

Run34 

BRD BL 

CCFN30 

CCFN30 

CCFN1 

EBGFN30 

EBGFN1 

| Rare Event 

| Run35 

| EBG1 

| EBGFN1 

1 PRl 

| PRFN1 ] 

| EBGFN3 i 

1 PR3 1 


60 




Appendix B: Post-Run Questionnaires 


The subject pilot completed a questionnaire (see below) at the end of each evaluation run. 
Mental Demand, Physical Demand, Temporal Demand, Performance (Pilot), Effort, and 
Frustration were all TLX measures that evaluated workload. Demand and Supply of Attentional 
Resources and Understanding of Situation values were combined to derive the SART rating. And, 
Level of Terrain Awareness and Stress were two other independent measures that were collected. 

MENTAL DEMAND 

1 i 1 . 1 i 1 i ! i 1 . 1 i 1 

> 1 , l > l 


DEMAND ON ATTENTIONAL RESOURCES 

Low 

PHYSICAL DEMAND 

1 , 1 i 1 , 1 i 1 , 1 i 1 , 1 

High 

. 1 l I I I 

1 

1 1 

Low 

High 

SUPPLY OF ATTENTIONAL RESOURCES 

Low 

High 

1 

1 1 

TEMPORAL DEMAND 

1 I 1 1 1 1 1 1 1 , 1 1 1 1 1 

. 1 l i 1 I 

Low 

High 

Low 

High 


UNDERSTANDING OF THE SITUATION 

PERFORMANCE 


l 

1 1 

1 , 1 , 1 , 1 i 1 i 1 i ! i 1 

1 1 1 I I I 

Low 

High 

Good 

EFFORT 

1 I 1 < 1 < 1 < 1 I i , 1 : i 

Poor 

, l , i , i 

l 

LEVEL OF TERRAIN AWARENESS 

1 1 

Low 

FRUSTRATION 

1 i 1 i 1 i 1 i 1 i 1 i 1 i 1 

High 

jJ 1 1 1 1 

Low 
| | 

High 

STRESS 

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 

Low 

High 

Low 

High 


Workload: 

Situational Awareness: 

TLX -Blue 

SART - Red 

Stress - Purple 

TA - Green 
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RUN QUESTIONNAIRE KEY: 


Title 

MENTAL DEMAND 


PHYSICAL DEMAND 


Descriptions 

How much mental and perceptual activity 
was required (e.g., thinking, deciding, 
calculating, remembering, looking, 
searching, etc.)? Was the task easy or 
demanding, simple or complex, exacting or 
forgiving? 

How much physical activity was required 
(e.g., pushing, pulling, turning, controlling, 
activating, etc.)? Was the task easy or 
demanding, slow or brisk, slack or 
strenuous, restful or laborious? 


TEMPORAL DEMAND How much time pressure did you feel due to 

the rate or pace at which the tasks or task 
elements occurred? Was the pace slow and 
leisurely or rapid and frantic? 


PERFORMANCE How successful do you think you were in 

accomplishing the goals of the task set by 
the experimenter (or yourself)? How 
satisfied were you with your performance in 
accomplishing these goals? 

EFFORT How hard did you have to work (mentally 

and physically) to accomplish your level of 
performance? 


FRUSTRATION LEVEL How insecure, discouraged, irritated, 

stressed and annoyed versus secure, 
gratified, content, relaxed and complacent 
did you feel during the task? 
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Please rate your overall impression of the scenario in terms of how much attention and effort was 
required to perform the scenario successfully. Things to consider are the degree of instability, 
complexity, and variability that you perceived while flying the scenario. 

DEMAND ON ATTENTIONAL RESOURCES 


Low High 

Please rate your overall impression of the scenario in terms of the amount of “spare” attention to 
give to other tasks. Was 100% of your attention directed towards successfully completing the 
scenario? Or, could you have completed other sub-tasks while flying the scenario? Things to 
consider include your level of arousal, level of concentration, and if you attention was divided 
across many sub-tasks. 

SUPPLY OF ATTENTIONAL RESOURCES 


Low High 

Please rate your overall understanding of what was happening with the aircraft. Mark on the line 
below the degree to which you felt confident that you were aware of the elements in your 
environment. Things to consider include the level of information quantity and quality as well as 
familiarity that you felt you had with what was taking place during the scenario. 

UNDERSTANDING OF THE SITUATION 



Low High 


Please rate you overall understanding of the terrain environment you were operating within. 
Things to consider for this response are how comfortable were you with your terrain awareness. 


LEVEL OF TERRAIN AWARNESS 



Low High 


Please rate your overall level of stress that you experienced while completing the experimental 
test run. 

STRESS 



Low 


High 
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Appendix C: Post-Block Questionnaires and Pilot Comments 


After each maneuver block, the subject pilots were asked to fill out a questionnaire that consisted 
of text questions regarding subject pilot FOV strategy, performance and terrain awareness ratings 
for each display concept, and general questions about the use of the primary and strategic flight 
displays, as well as an SA-SWORD (attached at the end of this appendix). Questions are shown 
in italics. Explanations of pilot comments were added by the authors of this document and are 
indicated in brackets in the text below. SI through S27 signifies subject pilot 1 through subject 
pilot 27. 

FOV Questions 

1. Overall, which field-of-view setting did you most prefer during the experimental trials? 

2. Overall, which field-of view setting did you least prefer during the experimental trails? 



High Altitude 

Low Altitude 

Approach | 

Subject 

Most Prefer 

Least Prefer 

Most Prefer 

Least Prefer 

Most Prefer 

Least Prefer 

1 

60 

22 

60 

22 

60 

22 

2 

60 

30 

90 

30 

90 

22 

3 

90 

30 

90 

30 

90 

30 

4 

60 

22 

60 

22 

90 

22 

5 

60 

22 

60 

22 

60 

22 

6 

90 

22 

90 

22 

90 

30 

7 

60 

90 

60 

22 

90/22 

30/60 

8 

60 

30 

60 

30 

60/30 

90 

9 

60 

22 

60 

90 

90 

60 

10 

60 

22 

60 

22 

90/60/22 

30 

11 

60/90 

22/30 

60 

90 

90/60/22 

30 

12 

30 

90 

30 

22 

30 

90 

13 

90 

30 

60 

30 

60 

30 

14 

60 

22 

60 

30 

60 

30 

15 

60 

22 

60 

22 

60/22 

30 

16 

60 

22 

60 

22 

60 

22 

17 

90 

22 

60 

22 

90/30 

22 

18 

60 

90 

22 

90 

90 

30 

19 

60 

22 

60 

22 

90/60 

22 

20 

60 

30 

60 

30 

60/22 

90 

21 

60 

22 

60 

22 

90 

30 

22 

60 

22 

30 

90 

60 

22 

23 

60 

22 

60 

22 

60 

22 

24 

60 

22 

60 

22 

90 

22 

25 

60 

30 

60 

30 

60 

22 

26 

60 

22 

60 


60 

90 

27 

30 

90 

60/30 

90 

90/22/30 

60 

Max 

90 

90 

90 

90 

Too varied 

90 

Min 

30 

22 

22 

22 

Too varied 

22 

Average 

62.3 

34.3 

59.7 

34.9 

Too varied 

38.2 

STDEV 

14.5 

24.5 

15.4 

26.5 

Too varied 

24.6 

Preferred 22° 

0 


3.8% 


Too varied 


Preferred 30° 

7.6% 


7.7% 


Too varied 


Preferred 60° 

76.9% 


76.9% 


Too varied 


Preferred 90° 

15.4% 


11.5% 


Too varied 



To come up with the percentages, the author excluded those entries that did not actually select 
one single preference. 
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3. Please provide the reason for the difference between your least and most preferred FOV 
choice: 

High Altitude: 

S 1 : More detail visible at 60 in periphery; less detail visible, too sensitive at 22. 

S2: Straight and level, preferred 22. Turning, prefer 90, with the horizon and ground clearance. 
S3: 90 - was sharper, gave a more detailed image immediately in front of nose. 30 - depends on 
what aircraft is doing (banks, descents, etc.). During straight and level flight, changing FOV is 
not a major impact on flying. Changing it during turns creates a different visual display which is 
not what was displayed on the previous FOV (before it was switched). When, for example, using 
a 90 FOV display, and cycled to a 30 or 60 FOV, the heading and altitude were considerably off 
from the 90 FOV. 

S4: 60 represented the forward view the best. 22 was disorienting as well as the 30 view. 90 
view gave the impression that I was flying “downhill” and was somewhat disorienting at cruise. 
S5: 22 showed only blue sky - little or no ground. 60 showed ground, FN, obstructions, but did 
not have minification disadvantages of 90. 

S6: 90 seems to give better visibility under nose of aircraft. 

S7: 60 - was enough FOV to see the terrain, yet enough fine resolution on the symbology to keep 
within the parameters of test (mostly) 

S8: 60FOV gave better information for more accurate heading holding. 30FOV has no heading 
reference on horizon and felt like I was chasing the VV (to much movement relative to actual 
condition). I did like the 90FOV for determining my rate of turn. 

S9: At the altitude tested, the movement of symbology at 22 was too fast for my skill level, my 
flight control inputs seemed to be reactionary. (Used FOV to) forecast terrain ahead of aircraft. 
S10: Clarity and ease of mental processing - what was most intuitive. Preferred the 60 or 90 
because it gave best SA. 

SI 1: In the 60 and 90FOV mode, aircraft movements were not so jumpy, plus I had a better feel 
for what was underneath and around me. 22 mode was more like an attitude indicator than terrain 
awareness. 

S12: 90 is difficult to control the aircraft - noticeable deflection is major. 60 is marginally 
controllable, but gives good terrain data. 30 gave good control feel, but sacrificed terrain 
information, 22 seemed jumpy on control. 

S13: Better terrain awareness with the 90. The 30 wasn’t large enough or small enough. 

S14: En route 60 or 90 was preferred for wide angle view of terrain and easier to use heading 
scale. 22 makes horizon appear too high when en route, but good on final approach. 22 heading 
scale too wide on approach. FOV was changed for type of terrain presentation, but for better 
view of attitude en roue and on final approachA 

S 1 5 : Least was harder to control - less margin for error. It felt like I was chasing too much. The 
best choice gave me a better understanding of the overall picture. Descending turns were harder 
to control on FOV22. FOV60 seemed easier to see the overall picture. 

SI 6: For 60, I was capable of seeing more of my environment and more of the terrain. The 22 I 
feel would be better for short final on approaches. The 22 left out too much of the terrain, and 
really didn’t give a clear depiction of the flight environment. 

SI 7: Could see more area around me, even below me. Better detail. 

SI 8: 60 gave a good representation of my outside surroundings without getting too far out and 
for descents straight and level 22 FOV was good. 

SI 9: Resolution of heading tape and resolution of flight path vector to other symbology - 

balanced. Also, locates horizon higher. 90 seemed too compressed to trust. 60 was a good 
compromise. 

S20: In the en route portion of flight I found the wider view more useful (60 and 90). 30 was not 
used often. 22 was great for final approach into landing area. In a turn, wider angle was better, 
could see into the turn further ahead. Switching back and forth allowed me to see the terrain in 
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better detail. While en route, I liked to see more of the land layout and as I approached the field, I 
limited the viewing area so I could see the landing area in more detail. 

S21: Wider view equals more awareness of surroundings for emergency landing, etc., but 90 
seemed to show too much and made it seem like the plane was barely moving. 22 almost 
completely blocked out the surroundings. 

S22: The wider view provided greater terrain awareness. 

S23: 90 seemed too distant, less precise. 22 and 30 seemed to close - wanted to over control the 
plane. 

S24: 22 not useful for en route. 30 most useful for attitude capture (descents and turns). 90 best 
for terrain awareness. 60 good compromise between 30 and 90. 

S25: The 30 view appears “to far”. The 60 setting gave me the best trade-off in downward and 
forward view. 

S26: 60 seemed most realistic, more like view out of the window. 22 seemed to close. 

S27: Prefer 30 for the extra sensitivity of instruments (pitch, VV, etc). 90 least preferred since 
more difficulty controlling plane and “distortion” of terrain. Changed to smaller FOV (30) for 
turns, descents, and other maneuvers requiring added degree of controllability/sensitivity. 

Low Altitude: 

SI : 60 - good sensitivity, good scene content; 22 - too sensitive, insufficient scene content 
S2: 90 gives expanded views (terrain and shadows) 

S3: I could see near and far much more accurately with the 90FOV. Changing the FOV settings 
while straight and level from 90 to 30 created confusion on my ability to interpret numerical 
inputs on display. 

S4: 60 - best representation of forward view from cockpit. 90 gave the impression of being in a 
descent. 

S5: 60 provides acceptable FOV to include ground when straight and level at 6500’; 22 does not. 
22 does not show objects closer than about 2nm when level at 5000’. Concerned that 22 FOV is 
not providing picture of close in objects. Minified FOVs not providing sufficient degree of 
concern, objects seem further away, less threatening. 

S6: 90 gave me more of a view under the nose of the aircraft than did the narrow FOVs. 

S7: 60 - gives the best terrain SA for controllability of symbology (or plane). 

S8: 30 - horizontal movement was distracting. 30 FOV works well for turns and 90 FOV worked 
best for cruise. As the FOV increased, the movement on the screen slowed down (relaxed in level 
flight). For turns at 90FOV, I had better SA for the roll out heading and relative speed of the turn. 
S9: I didn’t like using 90 when the aircraft was closer to the ground. The wider the FOV it 
seemed more detailed (the terrain). 

S10: SA and excessive responsiveness at tighter/lower settings. 30 was ok at low altitude if the 
resolution was superior. Basically selected FOV based on altitude and instrument responsiveness. 
SI 1: Favorite FOV was 60 ~ 30 - it gave me a reasonable perspective of the terrain around me, 
as well as an easily interpretable horizon and use of my other flight instruments. The 22 was a 
little too narrowly focused for much of the airwork to allow me a good perspective of the terrain 
around me. 

SI 2: Controllability versus level of terrain information available. 22 seemed a little jumpy. 

S13: The 60 gave good depth and FOV while still giving a big picture. If you are going to use 30, 
you can get almost as good a view with the 60 and a better one with 22. 

SI 4: 60 and 90 are good for en route. 22 is good for approach. 30 wasn’t useful for either 
approach or en route. 

S15: Unity was too precise. 60 seemed to give you all you needed. 30 was good for straight and 
level to hold to your numbers. 60 and 90 seemed good for turns. 

SI 6: 60 provided wider view while 22 was too much of a close up. Only when there were no 
obstructions would I change to Unity. 
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S 1 7: Could see more detail, larger area. 

SI 8: 22 provided more precise information; 90 FOV was too wide. 

SI 9: Best compromise for out-of-window view and primary flight instrument resolution. 

S20: At a lower altitude, I can see more of the terrain at a higher FOV. When looking outside and 
seeing the visual cues, having a higher FOV allowed the display to look as I saw it outside. 
Initially, to see the terrain, I used the higher FOV and as you approach, for more detail, I like the 
lower FOV (to see detail of terrain). 

S21: 30 for descents as it is easier to maintain VV on a set descent angle; 60 for cruise, as it 
gives a much wider view and allows easy control; 90 makes it seem like things are unbelievably 
far away. 

S22: Preferred a closer range of view that still allowed a good sense of SA. 

S23: Same as before - higher than 60 seemed to be less accurate, lower was too close and fast, 
tending to overfly. 

S24: 22 not useful in en route environment. 30 best for attitude captures. 90 best for terrain 
awareness. 60 is good compromise between 30 and 90, and keeps from having to toggle back and 
forth. 

S25: The 60 setting seemed to me the best “forward” and “downward” view. 30 appears “too 
far”. 

S26: 60 seemed most realistic, didn’t change FOV, didn’t want to change sight picture. 

S27: Sensitivity of instruments, view of terrain. Preferred 30 for turns, descent for more crisp 
instrument response. Least preferred, 90, since couldn’t give good plane control. Liked 60 for 
cruise, more authentic portrayal of terrain. 

Approach: 

SI: 60 - more scene content, good sensitivity; 22 - not enough scene content, too sensitive 
S2: Tunnels are set ahead of time. Terrain is most visable in 90. Loc/GS are easy to use. Wanted 
to stay on 90 until middle marker, then follow runway to land, or go around. 

S3: 90 FOV was easier to fly straight and level; 30 required much more work. However, on 
approach, needed to change FOV from 90 to 30 to pick up GS/runway marks. 

S4: 90 allowed for a stabilized approach. At AGLs < 500’, the lower FOVs provided the site of 
the runway better without sacrificing stability. 

S5: 22 obscures near field objects (bottom of 22 FOV too far ahead of aircraft). 22 tunnel too 
hard to track. To optimize performance, used 90FOV in turn to intercept localizer. 

S6: 90 - more boxes to shoot for, best for extended final and during turns; 30 - not used. 22 - 
best for close in on final approach. 

S7: More boxes per mile on 90 for turn and initial line up. On final the fineness of the control on 
the 22 setting overrides the boxes per mile benefit. 

S8: During level flight, 90FOV boxes were too small to aim to. 60 and 30 were the best. I only 
switched around to help stabilize my view and stop correcting my corrections. If the FOV was 
bad or showed too much motion, (I would) move toward 90FOV (to) help stabilize picture. On 
approach (close-in), I zoomed in to see the field. 

S9: During the approach scenario, FOV90 worked best for controlling the VV through the boxes, 
then 22 worked best at or around the middle marker for lining up glide path indicator on runway. 
S10: 90 - turns, SA; 60, 30 - straight and level; 30 to 22 - for GS, final and touch down. Used 
FOV mostly for precision of flight instruments. 

S 1 1 : 90 for turns, since it gave me a better lead in. 60 for cruise and initial approach - allowed 
better view of upcoming terrain and 22 - 30 for near airport since it showed the runway 
environment. 30FOV to 22 too jumpy in cruise or initial approach. 

S12: Difficult to control at 90 setting. During approach, I like reducing from 60 for intercept to 
20 during approach, then 22 on short final to increase sensitivity of control display. 
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S13: The 60 gave enough boxes to make following the tunnel easy and is less sensitive. The 30 is 
too much in the middle, I would rather use the 60 or 22. The turns in the tunnel are easier with the 
90, and use 60 from the outer marker in, because of the amount of boxes shown, and the 
sensitivity. 

SI 4: 60 was good en route and approach until past the outer marker. Then 22 worked best to the 
runway. Liked tunnel effect provided by 60. Easier to navigate inside boxes and gave good terrain 
perspective. 

S 1 5 : I liked the 60 on initial approach and 22 on short final. I did not need the accuracy of 22 on 
initial approach, but did like accuracy on final. Did not like 90 as it had to many boxes. 60 
seemed just right for airspeed I was flying. 

SI 6: Good perception of terrain environment. 22 was too close for accurate correction. 

SI 7: Less confusing. More control of aircraft. Like 90 and 30 equally - 90 for initial approach, 
and 30 on final. 

SI 8: 1 preferred the 90LOV while on the localizer because I was able to see more of the “boxes” 
and the movement of the VV wasn’t as touchy. I switched to 22 FOV shortly after the middle 
marker, because it allowed a better view of the runway to line up the 3° line on the 1000’ mark of 
the runway. 

SI 9: Best balance between EOV for terrain and ease of using boxes. Using the boxes was easier 
at 90 during the turns. 60 or 90 worked fine for straight lines but 90 was better for turns because 
of more boxes. Could use 90 in straight but easier to see runway in 60 during approach. 

S20: Was easier to fly the boxes with the 60, but once at outer marker, 22 is the best - could hold 
VV to the touchdown zone. 90 just shows too many boxes. I found that changing the FOV during 
the flight too many times made me chase the boxes. So for outside the outer marker, 60 FOV was 
used, then about the outer marker, 22 was used. 

S21: For turns and general navigation, 90 because the boxes come quicker and are easier to fly 
through. Used 22 once runway was in sight, and focused on aiming points. 22 also gave better 
view of local topography around airport. 

S22: 60 provided good terrain information while also enabling me to maneuver through the 
boxes. The other views seemed either too close up or too far away. 

S23: Fligher than 60 was too far out. Closer was too close and precise, causing me to overfly the 
simulator. 

S24: 90 preferred for lowest spacing between successive tunnel rectangles. Also, good terrain 
awareness. 22 least preferred because limited FOV, limited utility. Second most often used was 
30 (for descent) because of increased pitch ladder resolution. Would have preferred 60 with 
tunnel spacing as for the 90. 

S25: The 60 view gives me the best trade-off on “forward view” and “below the nose” view. 22 
really has it limits with respect to what does the terrain look like below the airplane. 

S26: 60 - most realistic. 90 - seemed too far away. Like 22 on final approach, because gave 
better view of runway. 

S27: Chose FOV based upon size of tunnel boxes primarily (to give perspective for control of 
plane). I most preferred 90 during cruise and 22/30 during approach for this reason. Least 
preferred 60 since it was the mid-range between my preferred FOV settings. 

4. (Combined questions 4 and 5 on block questionnaire) Did any display condition you were 
flying change which FOV setting that you used? What was the reason for the use of different 
FOV settings for these display conditions? 

High Altitude: 

SI: Nope. I used (and preferred) the same FOV for all display conditions in this scenario. 

S2: Yes. Changed due to distractions of FN. Also, min color and horizon reference points. 

S3: No. N/A. 

S4: None - 1 preferred 60FOV on all display conditions. N/A. 
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S5: No. N/A. 

S6: No. Nothing in turns of display concepts. For maneuvering, possibly a wider FOV was more 
appropriate for a turn. 

S7: No. N/A since terrain was far away, no need to utilize FOV except for symbology. 

S8: No. N/A. 

S9: No. N/A. 

S10: No. N/A. 

SI 1: No. N/A. 

S12: Yes. Tended to ignore terrain on CCFN display, so kept in 30 or 22. With PR and EBG, 
used 60 more when level and 30 for maneuvers. So little terrain data with CCFN, was better to 
optimize control. 

S13: Yes. PRFN1, FOV60. Gave a better picture of all the terrain. 

SI 4: No. N/A. 

S15: No. N/A. 

SI 6: No. The only time I went to 22 was for aircraft correction. The 22 gave a more precise non- 
hesitant measure of control inputs. Control correction, only - had nothing to do with display 
concept. 

SI 7: Yes. Some of the displays looked better with the 60 - a little better with where I truly was 
at. More accurate feel for where I was. 

S18: Yes. With the higher detailed terrain, I used the 60FOV. I could see more of the terrain 
around me. 

SI 9: No. N/A. 

S20: No. N/A. 

S21 : No. I didn’t really make any connection between display and FOV. N/A. 

S22: No. N/A 
S23: No. N/A 
S24: No. N/A. 

S25: Yes. EBGFN30 and PRFN30. Just exploring the potential view. Although, at this altitude 
(9500’ to 8000’), it didn’t really matter. 

S26: No. N/A. 

S27: Yes. Less need to change to smaller FOV (30) with the PR1, since greater awareness of 
terrain. 

High Altitude Totals: 

Yes: 7 
No: 20 

Low Altitude: 

SI: Nope. Didn’t - I feel the FOV requirements change with the task, not these display 
conditions. 

S2: Yes. CCFN - would use 22 or 30, for the most part - close in for more precise flying, (may 
use 60 in the turn, to increase horizon). 

S3: No. N/A. 

S4: No. However, at low altitudes, the 30FOV setting became more useful than at the high 
altitudes. I still prefer the 60FOV setting. N/A. 

S5: No. Would scroll through FOVs to try to identify obstacles (towers, etc.) and to gain broader 
SA, but preference didn’t change with display concept. 

S6: No. Not in terms of display concepts. But, possibly a wider FOV is better during turns and 
while in high terrain. 

S7: Yes. High resolution EBG. With the high res. EBG, I was able to go to a 30FOV for better 
controllability of aircraft with symbology. 
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S8: No. N/A. 

S9: No. N/A. 

S10: Yes. CCFN only, used 90FOV. The better the resolution, the more comfortable I was with 
lower display angles. 

SI 1 : Yes. Baseline SVS. I was more interested in trying 30FOV or less, otherwise 1 wanted a 
slightly larger perspective of the upcoming terrain. The baseline SVS had less information being 
presented, so interpreting straight flight instruments was slightly easier at this level. 

S12: Yes. Felt more comfortable using 60 or 90 when using EBG versus PR. CCFN did not 
matter because so little to see on 60 and 90 views. Control conditions versus desire for terrain 
information. 

S13: Yes. PRFN1, FOV60. Depth of field and terrain awareness. 

S14: No. N/A. 

S15: No. N/A. 

SI 6: No. N/A. 

SI 7: Yes. I liked the 90 on some of the displays (EBG). Mountains were sharper on the EBG 
display at 60, could see more of them. 

S18: Yes. When I was using the CCFN30 and 1, the terrain information wasn’t very good from 
the display, so I used 22 FOV during CCFN30. So I could maintain my attitude or heading better. 
SI 9: No. N/A. 

S20: No. N/A. 

S21 : No. I didn’t really associate one with the other. N/A. 

S22: Preferred 22 or 30 with CCFN. Felt it gave me a better sense of awareness of where I was 
and control in terrain. 

S23: No. N/A. 

S24: No. N/A. 

S25: No. Just played with the settings to get an overall impression, although overall, the 60 
setting seems to work best for all 3 (CC, EBG, and PR). Exploring the settings, I found the 60 
setting comfortable for most scenarios. 

S26: No. N/A. 

S27: Yes. Could more accurately navigate with the PR, so sometimes went to higher (60) FOV to 
orient and navigate. N/A. 

Low Altitude Totals: 

Yes: 9 
No: 18 

Approach: 

S 1 : Nope. Used 30 a lot on final - better indication of vertical errors. 

S2: No. N/A. 

S3: No. N/A. 

S4: No. 1 selected the FOV that reduced the workload the most. 

S5: No. N/A. 

S6: No. N/A. 

S7: No. The terrain was not important except for the “no tunnel” case in which terrain awareness 
is important and then I’ll defer back to the en route answers. 

S8: No. N/A. 

S9: No. N/A. 

S10: No. N/A. 

SI 1: No. N/A. 

S12: Yes. CCFN. On CCFN, had to use lower settings to see and interpret the FN. 

S13: No. N/A. 
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S14: No.N/A. 

SI 5: No.N/A. 

SI 6: No.N/A. 

SI 7: No.N/A. 

SI 8: No.N/A. 

SI 9: No.N/A. 

S20: No.N/A. 

S21: No.N/A. 

S22: No.N/A. 

S23: No.N/A. 

S24: No.N/A. 

S25: No. I pretty much flew everything with the setting of 60. Played around with the other 
settings a few times, but none appeared appealing. 

S26: No.N/A. 

S27: No. Chose FOV primarily based on symbol (tunnel, VV, etc.) to maintain control. N/A. 

Approach Totals: 

Yes: 1 
No: 26 
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Low Altitude: 





Approach: 





Terrain Awareness: 

High Altitude: 
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Approach: 
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Approach Only: 

7. Please indicate whether the use of the tunnel significantly enhanced SA while making the 
approach versus no tunnel for the CCFN30 concept. If so, please provide a numerical rating of 
1 to 10 (as % ofSA improvement). 

S 1 : Definite improvement: 1 0 
S2: Yes: 10 

S3: The tunnel definitely made a huge difference in focusing on the area I needed to point the 
nose of the plane; took away a lot of processing normally required to fly: 10! 
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S4: The tunnel did enhance SA significantly. Increased SA by 40%. It reduced workload and 
required scan rate by 60-70%. 

S5: Not a significant improvement - slight improvement in SA: 1. 

S6: Yes. Significant enhancement: 9. 

S7: The tunnel was key in confirming SA when the blue/brown display was operating. It was 
still flyable, however, and the SA was confirmed by the diamonds for the loc and GS. The tunnel 
reduces the workload and becomes better situation to fly in (especially without VFR visibility). 

S8: 10!! 

S9: The tunnel, in IMC conditions, is a GREAT guide! As far as SA, i.e., location in relation to 
terrain, I don’t feel as though my awareness was enhanced, but guiding the aircraft safely to the 
ground is improved. 8. 

S10: 60% better. 

S 1 1 : Yes, greatly improved. 90%. 

S12: Yes. The tunnel definitely made a difference on both Loc and GS. 80% in these phases. 
Less important once on GS ~ 30%. 

S13: Yes, the tunnel helped. 10. 

S14: Tunnel made flying the approach tremendously easier than same display without tunnel. I’d 
give it a 10. 

S15: The tunnel was a great tool for setting up the approach it helped by at least 70% for SA. 
Not as helpful on short final - maybe 20% improvement on SA. 

SI 6: There was a 100% difference for accuracy. 

SI 7: Yes. 100%. 

SI 8: The approach with the tunnel increased my awareness from second to second, as opposed to 
not using the tunnel where I had to pay constant attention on how all the other parameters are 
doing (i.e., airspeed, altitude, rate of climb. Power RPM). 

SI 9: 7 or 8. Any predictive flight guidance overlaid on a terrain (with landmarks and obstacles) 
display would make SA much better. 

S20: Having the tunnel decreases your workload. You can spend more time on the other items. 
Less stress. 80% improvement with tunnel. 

S21: Yes. 8. The diamonds tell you, in your head, how far off/on course you are (you can do the 
math with the dots), but the boxes, at a constant size, show you where you are in relation to where 
you should be. 

S22: The tunnel made all of the difference in my performance. - 100% 

S23: Yes, it helped guide the approach. 8. 

S24: Yes. The tunnel greatly improved SA (10 on scale of 1-10). Ability to instantly perceive 
location in space relative to desired path without looking away from primary instruments 
(altimeter, airspeed indicator, etc.) was very valuable. 

S25: I would definitely give the tunnel a 10. This is a 100% improvement, beyond any doubt! 
Major improvement with respect to SA, much more time available to perform different tasks. 
Great concept! 

S26: Yes. 10. Felt overloaded (without tunnel). 

S27 : The use of the tunnel very significantly enhanced SA with CCFN by providing information 
for navigation and position. I would say it raised awareness by a factor of 9 (90%). 

AVERAGE: 8.5 (i.e., 85% increase in SA) 

MX20 Questions 

8. What type of information was provided specifically by the MX20? 

High Altitude: 

S 1 : Distance to waypoint. 

S2: Waypoints, Navigation (Nav) flight paths, baro, distance 
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S3 : Where my aircraft was in relationship to my waypoint and heading 

S4: Waypoint designation. No other information taken from the MX20 for these scenarios. 

S5: Waypoint arrival only - would have liked to use MX20 in terrain (normal) mode vice in 
terrain warning mode. 

S6: Abeam and turn points; high terrain 
S7: Waypoints and terrain. 

S8: Waypoint for turn (It was a distraction, actually). 

S9: Waypoints, checkpoints. 

S 1 0: Position and vector to/from waypoint 
SI 1 : When to initiate my turn towards the ridge. 

SI 2: Navigation, horizon, position with regard to turn point, altitude. 

S13: Navigation. 

S14: Waypoint, airplane location relative to waypoints, obstacles. 

S15: Waypoint information. 

S 1 6 : Heading information. 

SI 7: Where I was in relationship to the loc; how close I was to terrain. 

S 1 8 : Proximity to terrain. 

SI 9: Course and waypoints. Terrain too far below us to show up. 

S20: Visual distance to the next waypoint, and on a limited scale, for me, terrain information. 

S2 1 : Horizontal navigation to waypoints, ground proximity information. 

S22: No written comment. 

S23: Waypoint and flight path. 

S24: When to make the left tum/descent maneuver, orientation. 

S25: Heading, course and elevation (eye-down mode). 

S26: Waypoint - turn. Proximity to ground. 

S27: Primarily waypoint information (when to turn). Don’t recall having seen terrain information 
at that altitude. 

Low Altitude: 

SI : Terrain - to a little extent, and waypoint passage 

S2: Heading, towers, terrain, way points, en route flight, altitude, baro 

S3: Heading, altitude, terrain proximity, waypoint 

S4: Terrain awareness and waypoint identification 

S5: Only waypoint turn cue. Need good nav display information (i.e., MX20 in terrain (normal) 
mode) 

S6: Terrain information and abeam points. 

S7: Check data for the SVS altitude display when one existed. Primary source when SV display 
didn’t exist. Also, as waypoint routing information. 

S8: Waypoint only. Heading was displayed, but I did not use it. There was green for terrain, but I 
did not use it. The position of the display to the right was so far (distraction), I only needed turn 
information. 

S9: Aircraft altitude in relation to terrain, turning point. 

S10: Waypoint. Tertiary for terrain (cross-check only). 

SI 1 : Turning point. 

S12: Nav data. 

S 1 3 : Obstructions, “towers”. 

S14: Flight path awareness. 

S15: Waypoint information. 

SI 6: Heading, Attitude, and altimeter setting. 

S 1 7 : T errain, nav information. 
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SI 8: Terrain that was within so many feet of my altitude, location of radio towers, and 

waypoints. 

SI 9: Waypoint, heading, and terrain. 

S20: Next waypoint. Warnings of elevation data. 

S21: Waypoint location, ground proximity information. 

S22: Gave me indication of when to make course adjustments. 

S23 : Proximity waypoint information. 

S24: Heading, terrain proximity. 

S25 : Heading and birds-eye view of the terrain, course, terrain avoidance information. 

S26: Turn - waypoint. Proximity to terrain. 

S27: Terrain proximity (saw green for 500-2000’ clearance). 

Approach: 

S 1 : Some indication of terrain, nav position 

S2: Altitude, some terrain, but not down at ground level, heading, way point, path marked 
S3: Terrain proximity, heading , altitude, waypoint 
S4: MX20 not referenced when using the SVS 

S5: Terrain warning, waypoints. MX20 limited by requirement to be in terrain 10 mile mode. 

S6: Horizontal look at terrain, altimeter setting, altitude, turn points. 

S7: Very little extra information not provided by the tunnel. 

S8: Heading, terrain, waypoints. 

S9: Waypoints, checkpoints, terrain at your level and below. 

S10: SA, especially while flying on baseline (round dials) gauges. 

S 1 1 : Without the tunnel, provided course information on intercept and guidance for localizer. 

S 12: Waypoints, particularly without tunnel. 

S 1 3 : Navigation when using the baseline display. 

S14: Overall course SA. 

S 1 5 : Location of airplane relevant to the outer marker. Also to line up on approach without the 
box. 

SI 6: Altitude, heading, track, altimeter, terrain. 

S 1 7: Course track, outer marker, good SA. 

SI 8: The MX20 gave me good terrain information at first, but as I got closer to the ground, the 
only worthy information it gave me was locations of the towers and waypoints. 

SI 9: Terrain and course plus waypoints and outer marker. 

S20: Initially, just distance to waypoints, then terrain warnings. 

S2 1 : Surrounding GP W S . 

S22: MX20 gave me information about any course/approach and terrain avoidance (although I 
did not use it much for this with SVS because it added to my workload - SVS gave me the terrain 
information I needed.) 

S23 : I barely looked at it. 

S24: Desired flight path (horizontal). 

S25 : Heading (course) and terrain avoidance. 

S26: Turn - waypoint. Terrain. 

S27: Route planning (waypoints) and terrain proximity/avoidance. 

9. What type of information was provided specifically by the SVS display? 

High Altitude: 

S 1 : Everything else - tactical and (where given) strategic information 

S2: Heading, airspeed, alt, vertical degrees, horizon, pitch, bank, side slip flag, acceleration 
arrow 
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S3: Type of terrain, the higher and lower elevations contracting dramatically, valleys and peaks 
were very easy to detect. 

S4: Horizon information was the most helpful when initially entering IMC. 

S5 : Depending on display type, terrain SA 

S6: Terrain features, attitude information, airspeed, altitude, trend information 
S7 : Flight parameters and terrain. 

S8: All performance information as well as heading. There was terrain information that I did not 
give much time to because terrain was not a factor at that altitude. 

S9: Flight information, airspeed, altitude, etc. 

S10: All gages and terrain separation. 

SI 1 : Terrain awareness, heading, altitude, speed, and direction in which the aircraft is heading 
(i.e., VV indicator). 

S12: Primary navigation information, terrain awareness, major features of terrain. 

S13: Navigation, SA, airspeed, attitude, altitude, etc. 

S14: Terrain contour, runway surface, height of terrain in relation to current airplane altitude. 

S 1 5 : Altitude, speed, heading, bank, pitch. 

SI 6: Heading, pitch, and bank. Airspeed, altitude, VSI. 

SI 7: Location to terrain, SA - displays with terrain shown. 

SI 8: Terrain and SA. 

SI 9: Primary flight information and terrain and obstacles. 

S20: Terrain layout and profile, airspeed, altitude, heading. 

S21: Clear-sky view, airspeed, heading, bank, VSI, altitude, accel/decel, GS, VV 
S22: No written comment. 

S23 : Heading, altitude, attitude, and airspeed. 

S24: Attitude, airspeed and trends, altitude, heading, vertical speed. 

S25: Airspeed, altitude, rate of climb/descent, heading, slip/crab, airspeed trends (accelerate, 
decelerate, or constant speed). 

S26: Primary flight instruments. 

S27: SVS provided information on the attitude of plane, its horizontal and vertical speeds, VV 
information - trends. 

Low Altitude: 

SI : Tactical information and terrain awareness 

S2: All flight information, terrain, leads or trends 

S3 : Heading, altitude, airspeed, bank angles, pitch 

S4: Artificial horizon, terrain awareness, and primary flight instruments. 

S5: Terrain awareness, attitude information, airspeed, altitude, and heading. 

S6: Terrain trend information, airspeed, altitude, altimeter setting. 

S7: Terrain awareness and altitude plus flight parameters. 

S8: Everything I needed, even though I would like RPM, to navigate and avoid terrain. 

S9: All flight information as well as relief/contour of the ground. 

S10: Terrain separation! All instrument functions. 

SI 1: Terrain awareness, attitude, speed, direction. 

S12: Control (instruments) data, terrain details and features (rivers, towers), terrain data well 
below aircraft. 

S13: SA, airspeed, attitude, altitude, etc. 

S14: Flight parameters (airspeed, altitude, attitude, heading). 

S15: Heading, bank, altitude, attitude, slip, air speed. 

SI 6: Airspeed, attitude, heading, altimeter setting, bank angle and picth angle. 

SI 7: Terrain, all normal information - heading, altitude, etc. 

SI 8: Airspeed, altitude, bank angle, heading, rate of descent, VV. 
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SI 9: Primary flight instruments, terrain, and obstacles. 

S20: Terrain avoidance, aircraft performance and control. 

S21: Speed, VS1, heading, altitude, speed trend, VV, bank angle, turn-slip. 

S22: SA, elevation of the terrain, landmarks. 

S23: Attitude, altitude, airspeed. 

S24: Attitude, airspeed and trend, altitude and trend, terrain information. 

S25: Airspeed, altitude, roll angle, slip, climb/descent, and heading. 

S26: Primary flight instruments. Realistic look of terrain. 

S27: Got the necessary information for handling the plane and maintaining navigation. 

Approach: 

S 1 : Tactical guidance, terrain information 

S2: All flight instruments. Terrain on EBG, PR, but not on CC. Cities on PR. 

S3: Heading, speed, bank angle, altitude, power setting, GS 

S4: All information provided that was needed to make the approach safely 

S5: Terrain awareness, attitude information, altitude, airspeed, heading. 

S6: Altitude, heading, terrain awareness, tunnel information, trend information. 

S7: Everything needed to fly the approach: altitude and course with tolerances. 

S8: All information for flight except navigation. However, on approach, the nav data in the form 
of the boxes was GREAT! 

S9: Primary flight information, airspeed, heading, etc., as well as terrain display. 

S10: Flight instruments, VV (loved that! !), detailed terrain information. 

SI 1: Attitude, altitude, speed, VV (very helpful), VSI. 

S12: Terrain, primary instrument data. 

S13: SA, terrain, navigation, airspeed, altitude, attitude. 

S14: Highway in the sky. 

S15: Terrain, altitude, airspeed, bank angle, and SA (where you were on GS). 

SI 6: Terrain, heading, altitude, airspeed coordination, vertical speed, attitude, bank and pitch 
angle. 

S 1 7: Route and approach information, guidance for approach. 

SI 8: Tunnel location (localizer), terrain information, and flight parameters. 

SI 9: Flight guidance and a view out of the window display. 

S20: Course guidance and aircraft performance. SA and terrain avoidance. 

S21 : All information on standard 6 gauges, except for RPM! Plus VV and acceleration trend. 

S22: Approach route and terrain. 

S23: Everything else - altitude, attitude, airspeed, VSI. 

S24: Everything else (airspeed, heading, altitude, etc.). 

S25: Airspeed, heading, and altitude, and specifically helpful is the caret to maintain constant 
airspeed. 

S26: Primary flight instruments, terrain. 

S27: Attitude, speed, altitude, and trend information, as well as enhanced navigation with the use 
of tunnels. 

10. What information was provided by both the MX20 and the SVS displays? 

High Altitude: 

SI: Nothing 

S2: Heading, General terrain information. 

S3: heading 

S4: I used the MX20 primarily for waypoint identification. The SVS was used to maintain 
aircraft control to PTW requirements and to stay upright. 

S5: Terrain warning (N/A for this scenario - high altitude) and terrain SA. 
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S6: Presence of high terrain. 

S7: Cross-contamination on terrain. 

S8: The MX20 only added the nav information (waypoints) 

S9: No written comment. 

S10: Cross check of heading. 

S 1 1 : Heading indication? 

S12: Altitude, towers within 2,000’. 

S13: Terrain. 

SI 4: Obstacles, terrain altitude in relation to aircraft. 

S15: None. 

SI 6: Heading. 

SI 7: Terrain. 

SI 8: I was able to have the entire situation in view with both displays. 

SI 9: Heading. 

S20: MX20 - next waypoint. Used SVS for primary flight information and just glanced at MX20 
for next waypoint. 

S21 : Ground proximity information, though through different methods. 

S22: No written comment. 

S23: No written comment. 

S24: Orientation/heading. 

S25: Heading (although the MX20 doesn’t respond as fast as changes in heading). 

S26: Relationship to the ground. 

S27: Together they provided an awareness of terrain proximity and position/awareness. 

Low Altitude: 

SI : Terrain information (to some extent) 

S2: Heading, altitude, terrain clearance 
S3 : Heading and altitude 
S4: Terrain awareness 

S5: Both ? Terrain warning also provided by the MX20, but not used much. 

S6: More comprehensive idea of terrain awareness. MX20 did give you horizontal view of 
position. 

S7: The full picture of a flight with redundancies on terrain awareness. 

S8: Complete SA. The MX20 is a lot of hardware for one function (nav data). 

S9: The two displays were essential in base-line and constant-color for terrain avoidance. 

S10: Heading macro, large scale SA. 

S 1 1 : Heading information, some terrain awareness. 

S12: Altitude, terrain data approaching aircraft altitude. 

S 1 3 : T errain and navigation. 

SI 4: Terrain, obstacles, heading (approximate heading on MFD). 

S15: None 

SI 6: Altimeter setting, heading, and altitude. 

SI 7: Terrain. 

SI 8: I was able to know what the aircraft was doing, where it was in relation to terrain, and also 
with relation to ROA airport. 

SI 9: Terrain and obstacles, heading. 

S20: MX20 gives distance to waypoints, warnings of terrain based on altitude. In the lower 
DEMs, or even the higher DEMs (Chelton), used the MX20 more for terrain avoidance than just 
SVS at higher DEM. 

S21 : Ground proximity, but through different methods. 

S22: Combined, they gave a heightened sense of SA. 
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S23 : Proximity waypoint info and attitude, altitude, and airspeed. 

S24: Terrain information and heading. 

S25: Heading. 

S26: Proximity to terrain. 

S27: Together, they provided terrain avoidance information, as well as SA. 

Approach: 

S 1 : Terrain (to some extent) 

S2: Altitude, headings, some terrain 
S3: Altitude, heading 

S4: In the baseline scenario, the MX20 was used for increased SA. The SVS was the primary 
source of information. 

S5: No written comment. 

S6: Combination instruments for SA. 

S7: See above answers. 

S8: All necessary flight information. 

S9: All flight information you would need in IMC. 

S10: Bearing. Big picture terrain scenario. 

S 1 1 : Localizer information once established and heading indicators. 

S 12: General terrain information, flight path (with tunnels). 

S13: I relied almost entirely on the SVS for all the information that I needed. 

S14: Terrain, obstacles, and flight planned course. 

S 1 5 : Y our location on approach. 

SI 6: Attitude, heading, altimeter. 

S 1 7: Course track. 

S 1 8 : T errain information. 

SI 9: Lateral approach course guidance information. Terrain awareness, except the MX20 terrain 
awareness resolution is poor. 

S20: Course guidance and aircraft performance. SA and terrain avoidance; the MX20 gave 
waypoint information for distance plus terrain warning. 

S21: Local GPWS. 

S22: Heading (MX20); terrain and approach (SVS). 

S23: Mostly used SVS. 

S24: Desired flight path (when tunnel was active). 

S25: Heading and terrain awareness. 

S26: Terrain. 

S27: Together, they gave navigation, terrain avoidance, and SA. 

11. What situations (e.g., approach, route-planning, non-normals, terrain avoidance, IMC, 
etc.) would the MX20 provide most of the useful information? 

High Altitude: 

SI: Terrain, route planning, nav following, strategic info, non-normals or emergencies related to 
nav. 

S2: Way points, distance, terrain, compass card (e.g., 90° or 180°) 

S3: Route planning 

S4: Approach, route planning, terrain avoidance. 

S5: Route planning, non-normals, strategic SA, navigation, terrain warning. 

S6: Route planning and non-normals; terrain avoidance. 

S7: Flight parameters and terrain. 

S8: Approach, route planning, emergency airports, (an electronic set of charts, approach plates 
and facilities directory). 
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S9: Route planning, IMC. 

S10: Approach, airport or obstruction proximity, waypoints. 

S 1 1 : Did not use the MX20 except for the turning point. 

S12: Route planning, non-normals (deviations), terrain avoidance, emergency route execution 
from high enough, en-route navigation. 

S13: Terrain avoidance. 

Si 4: Awareness of flight planned route and height above terrain (plan views) along flight 
planned route. 

S 1 5 : Route planning and approach information. 

S 1 6 : Route Planning. 

SI 7: Navigation. 

SI 8: Terrain avoidance, IMC, approach information. 

SI 9: Planning, course information including course-line, approach guidance (lateral), terrain 
awareness. 

S20: Possible warnings of terrain at altitudes. Visual cures of altitude proximity with terrain. 
MX20 along with SVS, once a pilot got used to working with both, could provide better terrain 
awareness. 

S21: En route navigation, definitely. You should already know all the information about your 
route beforehand, and use the MX20 to stick to that route. Terrain avoidance though for 
deviations from course. 

S22: No written comment. 

S23: Route planning and terrain avoidance. 

S24: None. MX20 would be useful for monitoring approach phase, but SVS would be primary 
instrument. 

S25: Terrain avoidance. 

S26: Route planning, terrain avoidance. 

S27: MX20 would be most useful for terrain avoidance and route planning (waypoints). 

Low Altitude: 

SI: Route planning, strategic en route (nav planning, route changes, nav errors, etc.)\ 

S2: En route planning, waypoints, back-up for heading and altitude, baro settings, emergencies. 
S3: Waypoint, nav, terrain avoidance 

S4: Route planning, approach, terrain avoidance, IMC, and waypoint identification 
S5: Route planning, non-normals, Terrain awareness, strategic planning. 

S6: Route planning, non-normals, terrain. 

S7 : Route planning. 

S8: Route navaids, emergency information. 

S9: Route planning, terrain avoidance. 

S10: Route planning, approach, airspace, big picture awareness. 

S 1 1 : Route planning, some approach information. 

S12: Route planning, emergency “escape” execution, terrain avoidance, en route navigation. 

S 1 3 : Route planning. 

S14: Flight path awareness. MFD is good back up for terrain and obstacles shown on the PFD. 
MFD plan view of flight planned course and airplane relationship to terrain, obstacles and course 
SA shows a larger distance than available on PFD which is very good. 

S15: Ground speed, route planning, approaches. 

SI 6: Route planning, terrain avoidance. 

S17: Terrain avoidance. 

SI 8: Route planning, IMC, terrain avoidance. 

SI 9: En route and some approach for position and terrain awareness. 

S20: Approach, and terrain avoidance. 


85 



S21: En route guidance, approach guidance, advanced Ground Proximity Warning System 

(GPWS). 

S22: Approach and route planning. 

S23: Terrain avoidance, IMG, route planning. 

S24: Approach or route planning (providing steering cues). 

S25: Terrain avoidance. 

S26: Approach. 

S27: MX20 would provide useful information on approach when getting especially near to the 
ground (terrain avoidance). 

Approach: 

S 1 : Route planning, nav status, long term strategic information 
S2: Waypoints, turning points, IMG, leg information, headings. 

S3: Waypoint, terrain avoidance 
S4: Route Planning. 

S5: Route Planning, approach, lateral SA, non-normals, terrain warning, strategic planning 
S6: Non-normals, IMC 

S7: Route planning and non-normals, but on the last run today (Rare Event), I did not think to 
use it because I never used it for the approaches, and forgot it was there. 

S8: Route planning. I did not use it much, if at all. 

S9: Route planning, terrain avoidance, IMG. 

S10: Planning, airspace, cross-check. 

S 1 1 : Route planning, possible initial approach. 

S12: En route, planning initial approach fix for navigation. 

S 1 3 : En route information. 

S14: Route planning, terrain avoidance (colors) and SA of route of flight, cue to look for 
obstacles on PFD. 

S15: Route planning and approach. Also, your location at any given time. 

SI 6: Route planning, non-normals, terrain avoidance. 

SI 7: Approach. 

S 1 8: En route, waypoint information, IMG. 

SI 9: MX20 would have been useful for approach in map mode with better range scale. Also, en 
route is great place for map. 

S20: MX20 gives a graphical representation for your toue, while the SVS gives you course 
guidance. The MX20 works well for route planning and a backup for terrain avoidance. 

S21: En route guidance, guidance in a go-around situation in IMG, and VMC, assist in finding 
approach fixes. 

S22: Approach and route planning. With more experience, I would appreciate it better for the 
other areas listed. 

S23 : Route planning, terrain avoidance. 

S24: Route planning. 

S25: Approaches, terrain avoidance, and unexpected situations, such as a go around, and locating 
emergency landing sites. 

S26: Terrain avoidance, IMG. 

S27 : MX20 would be useful for terrain avoidance and route planning (waypoints, outer marker, 
etc.). 

12. What situations (e.g., approach, route-planning, non-normals, terrain avoidance, IMC, 
etc.) would the SVS display provide most of the useful information? 

High Altitude: 

S 1 : Terrain avoidance, tactical problems, IMG, near term planning 
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S2: Heading, airspeed, altitude, terrain awareness in IFR (horizon). 

S3: Terrain avoidance 

S4: Approach, terrain avoidance, IMC. SVS would greatly reduce the risk of rolling inverted 
while adjusting to unexpected IMC conditions. 

S5: Approach, terrain avoidance, IMC. 

S6: Approach, terrain avoidance. 

S7: Terrain avoidance and IMC. 

S8: Just not route planning. Awesome! I only want nav waypoints. That covers en route info and 
non-normals. 

S9: All aspects of the flight. 

S10: Approach, terrain avoidance, IMC, route planning, waypoints. 

SI 1 : SVS display most useful in approach, non-normals, terrain avoidance, and IMC. 

S12: Approach, terrain awareness (ahead), IMC (particularly if VFR pilot), emergency route 
planning, basic maneuvering. 

S13: All of the above. 

SI 4: Control of aircraft attitude when IMC and visual of airport for approach 3-D view of terrain 
and height above terrain is about equal to terrain presented on MX20. 

S15: Non-normals, terrain avoidance, cruise flight, climbs and descents, approach and landing. 

SI 6: Approach, IMC, terrain avoidance, non-normals. 

SI 7: Approach, Terrain avoidance, IMC. 

SI 8: Approach, IMC, and en route. 

SI 9: Approach and departure terrain and obstacle awareness. 

S20: Terrain avoidance - being able to see the terrain and moving aircraft to avoid the terrain. 
Approaches to airports. Liked the VV function. I can see that once a person gets used to it, it 
would make possible a more stabilized approach. 

S21: Approach and terrain avoidance and any type of maneuvering. Straight and level flight 
shouldn’t require this. 

S22: No written comment. 

S23: Approach, IMC, terrain avoidance. 

S24: All - SVS provides primary required information (attitude, airspeed, etc.) in all phases of 
flight. 

S25: IMC, 

S26: Approach, route, terrain, IMC, 

S27: SVS would be most useful for approach, IMC, terrain avoidance, and SA. 

Low Altitude: 

SI : En route tactical - rare events (altimeter errors, etc.) 

S2: Emergencies, en route terrain avoidance, IFR. 

S3: terrain avoidance on descents, terrain reference points. 

S4: IMC, terrain avoidance, approach 

S5: Approach, IMC, Terrain avoidance, tactical planning. 

S6: Approach, terrain avoidance, IMC, 

S7: Terrain awareness, IMC, 

S8: All areas, except route planning (except power setting). The SVS was complete. One thing 
(that might be beneficial), like VOR, a heading bug for horizontal alert. 

S9: All of the conditions mentioned in the question. 

S10: Terrain avoidance, IMC, approach, route planning, wind correction. 

SI 1 : Non-normals, terrain avoidance, IMC, and I believe approach information would be 

extremely valuable. 

S12: Terrain awareness, IMC, VFR nav (on hazy days), non-normals, emergency “escape” 
planning. 
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S 1 3 : Approach, non-normal. 

S14: All flight situations. MFD simply provides plan view terrain and obstacles that are seen on 
the PFD, but at larger range. PFD and MFD compliment each other quite well to give pilot a 
more complete picture. 

S15: Terrain avoidance, approach, IMC, climbs, descents, cruise. 

SI 6: Approach, non-normals, terrain avoidance and IMC. 

S 1 7: IMC, terrain avoidance, approach. 

SI 8: Approach, terrain avoidance, IMC. 

SI 9: Approach for guidance and reality check for terrain, obstacles, and runway. 

S20: Terrain avoidance, approaches, primary flight control of aircraft. 

S21: Real-time GPWS, IMC descents/tums/climbs. 

S22: All of the above. I found that I referred less to the MX20 (except for approach, route 
planning) because it added to my workload. I believe that with more experience, however, it 
would become much more useful to me, especially for terrain avoidance. 

S23: IMC, terrain avoidance. 

S24: En route and approach terrain avoidance, plus any precision flying situations. 

S25: IMC, 

S26: Terrain avoidance, IMC, 

S27: SVS would provide useful information for most phases of flight (maneuvers, approach, 
navigation, IMC), since it shows what the plane is doing (attitudes, speeds) in relation to the 
terrain (especially with landmark features such as towers, airports, etc.). 

Approach: 

S 1 : Tactical information for all phases 

S2: Terrain avoidance, flying in IMC, tunnels on approaches, headings, airspeed on approach, 
altitude. 

S3: Approach with tunnel and GS markings. 

S4: SVS would be useful in all situations. It may be necessary to reference the MX20 for long 
legs of a cross-country. 

S5: Approach guidance, terrain avoidance, IMC, tactical planning. 

S6: Approach, terrain avoidance. 

S7: Approach, non-normals, terrain avoidance, IMC, cruise, etc. 

S8: By adding nav. data, it could be the sole display. 

S9: All of the above mentioned. 

S10: Terminal route, IMC, terrain avoidance, ILS and other approaches. 

S 1 1 : Approach, non-normals, terrain avoidance, general flight in IMC. The tunnel with the VV is 
very helpful for staying on course. 

S12: During approach, terrain avoidance particularly during the missed approach. 

S 1 3 : Approach and terrain avoidance. 

S14: Approach, IMC, terrain picture and good sense of terrain height in relation to airplane 
altitude. 

S15: Terrain avoidance, approach, and overall flying the airplane. It’s nice to have all your 
information on one screen. 

SI 6: Approach, non-normals, terrain avoidance, and IMC. 

SI 7: Approach, IMC. 

SI 8: Approach , terrain avoidance depending on which type of display is used, IMC. 

S19: Approach. 

S20: SVS is excellent for approaches, terrain avoidance and controlling aircraft in IMC 

conditions. 

S21: Approach definitely and go-arounds. Any maneuvers in IMC. Not really needed en-route. 
S22: All of the above. 



S23: IMC, terrain avoidance, approach, non-normals. 

S24: All other situations. Even with the tunnel not present, SVS display provides most of the 
information needed to fly. 

S25: Approaches, terrain avoidances, and IMC. 

S26: Approach, terrain avoidance, IMC, route planning. Need the tunnel for approach. 

S27: SVS would be most useful during approach, IMC, and situations requiring maneuvering of 
the plane. Also great for terrain avoidance since graphic representation of the ground is available. 

Other general comments: 

High Altitude: 

SI 9: This was high altitude, so terrain awareness wasn’t that noticeable or relevant. 

Low Altitude: 

S5: Pilot performance independent of display. 

SI 9: Standard gauges terrain awareness rating - may up it to a 1, using MX20. 

Approach: 

S5: Performance in meeting PTS criteria is independent of display. 

S7: Want to combine waypoints on SVS PFD - likes the secondary terrain awareness of the 
MX20, but would like for nav purposes the waypoints on the PFD. Also, on approach, maybe 
color code the tunnel to let you know where in the approach you are. Would be nice to hear the 
waypoint beep to remind you when the flight plan is about to change. 

S12: Point-of-View control: to shift VV closer to the center of display; also to move horizon line 
higher on screen for a better feel for terrain at the lower views. 

SI 9: Gave terrain awareness rating a 2 for standard gauges, even when including the MX20. 
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SA-SWORD 



Absolute 

(9) 

So 

Very 

Strong 

(7) 


Strong 

(5) 


Weak 

(3) 

cT 

Equal 

(11 

CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN1 










CC FN30 










CC FN30 










CC FN30 










CC FN30 










CC FN30 










CC FN30 










CC FN30 











Weak 


Weak 





EBG FN3 


Weak 


Weak 



PR FN30 
EBG FN3 
EBG FN1 
EBG FN30 

PR1 

PR FN1 
PR FN3 
PR FN30 
EBG FN3 
EBG FN30 

PR1 

PR FN1 
PR FN3 
PR FN30 
EBG FN30 
PR1 


PR FN1 



Weak 


Weak 



PR FN30 


Sample raw SA-SWORD data for one subject: 


SA-SWORD for SA for Pilot 1 (040302-040402) 

Judgment 

Matrices 


High Altitude 














A 

B 

C 

D 

E 

F 

G 

H 

1 

J 


GeoMean 

A 

1 

3 

-9 

-9 

-9 

-6 

-4 

-5 

-5 

-3 


0.015 

B 


1 

-9 

-9 

-9 

-7 

-7 

-7 

-7 

-5 


0.010 

C 



1 

3 

3 

7 

7 

6 

5 

9 


0.278 

D 




1 

6 

9 

8 

8 

8 

9 


0.268 

E 





1 

9 

8 

8 

8 

9 


0.187 

F 






1 

-4 

-5 

-4 

2 


0.030 

G 







1 

-3 

2 

8 


0.057 

H 








1 

6 

9 


0.086 

1 









1 

9 


0.050 

J 










1 


0.018 


A = CCFN1 
B = CCFN30 
C = EBG1 
D = EBGFN1 
E = EBGFN3 
F = EBGFN30 
G = PR1 
H = PRFN1 
I = PRFN3 
J = PRFN30 


Low Altitude 














A 

B 

C 

D 

E 

F 

G 

H 

1 

J 


GeoMean 

A 

1 

5 

-9 

-9 

-8 

-5 

-7 

-7 

-7 

-3 


0.014 

B 


1 

-9 

-9 

-9 

-7 

-7 

-8 

-8 

-5 


0.009 

C 



1 

-3 

2 

7 

8 

8 

8 

9 


0.228 

D 




1 

4 

9 

8 

8 

8 

9 


0.312 

E 





1 

9 

8 

8 

8 

9 


0.195 

F 






1 

-7 

-8 

-8 

-5 


0.019 

G 







1 

-2 

1 

7 


0.059 

H 








1 

5 

9 


0.084 

1 









1 

9 


0.057 

J 










1 


0.023 


A = CCFN1 
B = CCFN30 
C = EBG1 
D = EBGFN1 
E = EBGFN3 
F = EBGFN30 
G = PR1 
H = PRFN1 
I = PRFN3 
J = PRFN30 


Resulting Rankings 



High Alt 

Low Alt 

Approach 





1 

EBG1 

EBGFN1 

EBGFN1 

2 

EBGFN1 

EBG1 

EBG1 

3 

EBGFN3 

EBGFN3 

EBGFN3 

4 

PRFN1 

PRFN1 

PRFN1 

5 

PR1 

PR1 

PR1 

6 

PRFN3 

PRFN3 

PRFN3 

7 

EBGFN30 

PRFN30 

PRFN30 

8 

PRFN30 

EBGFN30 

EBGFN30 

9 

CCFN1 

CCFN1 

CCFN1 

10 

CCFN30 

CCFN30 

CCFN30 
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Approach 














A 

B 

C 

D 

E 

F 

G 

H 

1 

J 


GeoMean 

A 

1 

4 

-9 

-9 

-9 

-7 

-7 

-7 

-7 

-5 


0.012 

B 


1 

-9 

-9 

-9 

-7 

-9 

-9 

-9 

-4 


0.009 

C 



1 

-2 

6 

8 

6 

8 

9 

9 


0.259 

D 




1 

8 

9 

8 

8 

8 

9 


0.316 

E 





1 

9 

7 

7 

8 

9 


0.158 

F 






1 

-7 

-7 

-7 

-3 


0.021 

G 







1 

-4 

1 

7 


0.058 

H 








1 

7 

9 


0.092 

1 









1 

9 


0.053 

J 










1 


0.022 


A = CCFN1 
B = CCFN30 
C = EBG1 
D = EBGFN1 
E = EBGFN3 
F = EBGFN30 
G = PR1 
H = PRFN1 
I = PRFN3 
J = PRFN30 


Where: 

CCFN1 = Constant Color + Fish Net, DEM=1 

CCFN30 = Constant Color + Fish Net, DEM=30 

EBG1 = Elevation-Based Generic, DEM=1 

EBGFN1 = Elevation-Based Generic + Fish Net, DEM=1 

EBGFN3 = Elevation-Based Generic + Fish Net, DEM=3 

EBGFN30 = Elevation-Based Generic + Fish Net, DEM=30 

PR1 = Photo Realistic, DEM=1 

PRFN1= Photo Realistic + Fish Net, DEM=1 

PRFN3= Photo Realistic + Fish Net, DEM=3 

PRFN30= Photo Realistic + Fish Net, DEM=30 
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Appendix D: Final Questionnaire 

This is a summary of comments, based on written notes taken by the researcher during the interview (not 
actual transcripts). The “S” followed by a number signifies the specific subject pilot’s response. 

Display Texture Concept 

1. Please rank order your preference for which display concept that you would most like to have in 
your aircraft in marginal VFR or IFR weather conditions. Please make your choice based on 
texturing concept alone without consideration of the DEM. 


Subject 

EBG 

EBG+FN 

PR 

PR+FN 

CC+FN 

Baseline 

1 

2 

1 

4 

3 

5 

6 

2 

2 

4 

1 

3 

5 

6 

3 

1 

2 

3 

4 

5 

6 

4 

1 

2 

3 

4 

5 

6 

5 

1 

2 

3 

4 

5 

6 

6 

1 

3 

2 

4 

5 

6 

7 

2 

1 

4 

3 

5 

6 

8 

1 

2 

4 

3 

5 

6 

9 

2 

4 

1 

3 

5 

6 

10 

2 

1 

4 

3 

5 

6 

11 

5 

3 

2 

1 

4 

6 

12 

3 

1 

4 

2 

5 

6 

13 

4 

2 

3 

1 

5 

6 

14 

2 

1 

4 

3 

5 

6 

15 

1 

2 

3 

4 

5 

6 

16 

3 

4 

1 

2 

5 

6 

17 

3 

4 

1 

2 

5 

6 

18 

2 

4 

1 

3 

5 

6 

19 

3 

4 

1 

2 

5 

6 

20 

2 

1 

4 

3 

5 

6 

21 

4 

3 

2 

1 

5 

6 

22 

4 

3 

2 

1 

5 

6 

23 

3 

4 

1 

2 

5 

6 

24 

3 

4 

1 

2 

5 

6 

25 

2 

4 

1 

3 

5 

6 

26 

1 

3 

2 

4 

5 

6 

27 

3 

4 

1 

2 

5 

6 

Max 

5 

4 

4 

4 

5 

6 

Min 

1 

1 

1 

1 

4 

6 

Avg 

2.3 

2.7 

2.3 

2.7 

5.0 

6.0 

St Dev 

1.1 

1.2 

1.2 

1.0 

0.2 

0.0 
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SI: Low resolution CC worse than SVS baseline. 

S2: EBG - still good depiction, just lacking cities. PR over EBG because it shows cities and has more 
detail. 

S3: EBGs - colors stood out; able to pick out highlights, shadows, etc.; could pick out top of hill versus 
bottom of hill better than any other. PRs - just like OTW; Could pick out terrain features very easily; 
Color contrasts weren’t as good. CCFN - if had to use, could. It’s like bicycling across country versus 
riding in a Cadillac. But definitely would pick over traditional dials. 

S4: EBG by far the best, without FN. Very simple. Lighter at higher elevations really stands out. 

S5: FN didn’t mean anything - no influence, although may confuse it with roads. 

S6: No other comments. 

S7 : EBG gives more information about elevation than PR. FN is terrain dependent - told him what was 
at or away from him, made a more complete picture. Suggested the possibility of having cities outlined 
on EBG concepts. 

S8: No written comments. 

S9: No problem transitioning in and out of the cockpit with any of these. 

S 1 0 : No written c omments . 

SI 1 : Likes FN particularly on short final. CCFN texture gave more of contrast on manmade obstacles. 
S12: Texture preference actually depends on phase of flight, between PR and EBG. Down low and on 
approach, prefers EBG. En route, maybe PR to help pick out landmarks, from a navigation standpoint. 

S 1 3 : No written c omments . 

S14: EBG - easiest to perceive terrain profile. FN helps, gives North/South/East/West orientation. PR- 
not as easy to tell terrain profile, have to study it more. CC - really hard time noticing peaks in close 
proximity. 

SI 5: Can rely more on the shading of EBG than PR. PR gives color variation, but might misinterpret 
shading. CC - have to have the FN on. Baseline SVS - would take over what he has today. 

S 1 6 : No written c omments . 

S 1 7 : No written c omments . 

SI 8: Prefers SVS over standard gauges - eye movement is minimized, all information right there 

together, very easy to develop scan, gives your information overlaid on horizon. No FN versus FN - FN 
gets in the way. EBGFN tells you it’s flat at DEM=30, but shading gives you contour. 

SI 9: Likes PR a lot better than he thought he would. Didn’t think it would be much better than EBG. 
Maybe at higher altitudes, EBG is better. Approaches, PR wins hands down. 

S20: FN enhances 3D effect. EBGFN - easier to see elevation and different shades indicate different 
elevations. PR and PRFN - he doesn’t need to see everything that PR has. CCFN - where he’s flying, 
doesn’t like it. Concerned about flying over water. 

S21: Approach - EBGFN was as good as PR and PRFN, but at higher altitude, it wasn’t as good. PRFN 
- most realistic, easy to see terrain. FN enhances, but can still tell what’s going on with out the FN. 
CCFN is more useful than standard gauges. 

S22: No written comments. 

S23 : The biggest factor was discriminability of the FN that made the determination. Flad trouble seeing 
the FN with EBG and PR, but not with CC. Thought FN was crucial for CC, but could go either way with 
EBG and PR. 

S24: PR is the most natural to look at. With EBG, FN gave him different information than coloring; FN 
was more distracting here. For PR, really found FN more distracting than was useful; a little too artificial. 
CCFN - did not feel like he could get good information right away - not as intuitive. BRD - no terrain 
awareness. 

S25: Information wise, EBG, EBGFN, PR, and PRFN the same. Is more comfortable with PR. Don’t 
like the FN. 

S26: No written comment. 
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S27: Prefers EBG over CCFN because it’s more intuitive. PR - intuitive, same as flying “there”. It also 
has navigational quality. Flip-flopped on FN, sometimes FN conflicted with other features. BRD - lose 
so terrain, you have none at all. 

2. On a scale of 1 to 10, how much difference in your preference is there from choice 1 to choice 5. 1 
= almost none, 5 = Significant but not Strong, and 10 = Absolute. Please provide a reason for the 
numerical rating: 

SI: (answered choice 1 to choice 6 comparison) 10. (EBGFN compared to SVS baseline). With SVS 
baseline, no terrain awareness whatsoever. FN at low altitude gives you a great deal of information. 
Baseline - lot of information lacking, workload went up. 

S2: 10 (between PR and CCFN). Hands down, no contest. PR is unbelievable - just as if you’re looking 
out the window. 

S3: 10 - bicycling versus Cadillac. (EBG vs. CCFN) 

S4: 10 - CCFN difficult to pick out the heights on terrain. EBG, can definitely pick out peaks. 

S5: 8 - EBG very good SA with respect to terrain. Didn’t get that with CCFN. CCFN better than BSBG 
baseline, but hard to pick out terrain relief. 

S6: 8 - CCFN, doesn’t really differentiated elevations that well, like EBG does. 

S7: 10 - gets much more information out of EBGFN, can discern what’s far and near; much better 
depiction of reality than CCFN. 

S8: 10 - color variations on EBG was more alarming than CCFN. 

S9: 10 - actual representation of OTW with PR, as opposed to artificial with CCFN. 

S10: 7 - found information that he got from EBG was useful, intuitive, not a lot of processing. For the 
CCFN, had to process more to interpret. 

SI 1 : 9 - (PRFN as opposed to CCFN) FN - did like it for rolling hills. Almost like flying VFR. 

S12: 10 - (EBGFN over CCFN) CCFN required a lot of thought to 1) find FN, and 2) interpret. 

S13: 10 - (PRFN over CCFN). Much more realistic. Can see terrain a lot better. Indications of slopes 
and peaks. More definition. 

S14: 10 - (EBGFN over CCFN). EBG gives a realistic picture he can comprehend quickly. EBGFN is 
by far lowest workload. With the CCFN, he had to study it to interpret. 

S15: 10 - (EBG over CCFN). Color variation. You can just look at the picture and see why. 

SI 6: 10 - (PR over CCFN). Realism - being able to and accustomed to seeing things like that on the 
ground. 

SI 7: 10 - (PR over CCFN). PR just like OTW. CCFN caused dizziness. 

SI 8: 9 - Not totally absolute. Information that PR gives in case of engine out, can really pick terrain to 
go to. CCFN doesn’t give any terrain information. 

SI 9: 6 or 7 (6.5) - (PR over CCFN). CC - basically blue/brown. Probably a better way to implement 
and FN didn’t show terrain enough to work. 

S20: 10 (between EBG and CCFN). EBG so much easier. CC doesn’t give him a sense of urgency. 
With the rare event, he would have noticed the anomaly more quickly with EBG. 

S21 : 9. PR -much more useful. CCFN - didn’t seem very believable. 

S22: 10 (between PRFN and CCFN). Still don’t know where I am (CCFN) compared to PR. CCFN like 
a desert, but still better than 072. 

S23: 8. Like the PR because it looked VFR. Had significantly more trust with that concept compared to 
the CCFN concept. SA was enhance, giving it the rating of 8, due to the realistic and lifelike perspective 
afforded by the PR concept. 

S24: 9 (for just terrain). Have to concentrate to get any information on CCFN; not as much information 
per pixel. PR - sees contour and texture; used to processing, more intuitive. 

S25: 10. Likes PR a lot better than CCFN. Gives much more depth perception. CCFN gives 

information only in the near-ground. 
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S26: 9. EBG - coloring and shading enhance terrain. CCFN - terrain features are not enhanced and FN 
doesn’t help much. 

S27: 9. (between PR and CCFN). Never does a 10. PR is just so intuitive, can tell so much more. 

Average: 9.29. 

Fish Net 

3. Please indicate which of the following display concepts comparisons are superior in terms of 
enhancing situation awareness? EBG1 versus EBGFN1; PR1 versus PRFN1. 

S 1 : EBGFN 1 , and PRFN 1 . At low altitude, FN is really useful. FN at 3 arc-sec may be equivalent to no 
FN at 1 arc-sec, so may be able to sacrifice resolution (if you have a FN). Effect of FN more pronounced 
in EBG than in PR. 

S2: EBG1, and PR1. FN very distracting, takes away from elevations. Takes away from roads/rivers. 
Takes away from shadows. Can tell more height-wise without the FN. 

S3: EBG1, and PRFN1. FN is helpful when terrain is not flat. When terrain is flat, you might mistake FN 
for a road. For PR, the color contrast of lines using the FN helped identify peaks and valleys more than 
with out. 

S4: EBG1, andPRl. FN not a big factor in enhancing SA. 

S5: No preference. FN not enough contrast or something, but didn’t provide terrain relief. 

S6: EBG1, and PR1. No other comments. 

S7: EBGFN 1, PRFN1. FN gives more information. PR with FN begins to give you the elevation cues 
that you get with EBG. 

S8: EBG 1, and PRFN 1. No other comments. 

S9: EBG1, and PR1. No other comments. 

S10: EBGFN1, and PRFN1. FN was very intuitive. Not much processing. Flelps with steepness of the 
slopes. And, for PR texture, FN especially helped here - FN took a lot of the guessing out. 

SI 1: EBGFN1, and PRFN1. Not a strong preference, but at times did find that FN helped define terrain. 
For PR, because of the lack of contrast level, FN set ridges apart a little more for him. 

S12: EBGFN1, and PRFN1. EBG, not a strong preference. FN had more of an impact on PR. 

S13: EBGFN 1, and PRFN 1. FN just adds a little more depth perception. 

SI 4: EBGFN 1, and PRFN 1. FN is not at all distracting. Did not find ground following distracting. 

S15: EBG, and PRFN 1. Don’t need FN on EBG. FN gives a better interpretation of what’s happening 
on the PR. 

S16: EBG1, andPRl. Not having a FN makes it look more realistic. People may get confused with the 
FN when flying over rural areas, roads, etc. 

S17: EBG1, andPRl. FN distracting, looks like roads. 

SI 8: EBG1, and PR1. For EBG, FN was easily confused with the roads. For PR, FN helps during 
approach, but the quality of the DEMI is good enough, FN is not warranted. 

S19: EBG1, and PR1. For EBG, it’s not so objectionable to have FN, but would select the FN on EBG. 
FN doesn’t hurt that much, but would rather not have. Flave so much detail in PR that don’t need the FN. 
S20: EBGFN1, and PRFN1. EBGFN1 - FN gives sense of curvature. Can look out further. Can use to 
help judge distances. PRFN1 - especially here. PR is nice, but FN gives added information. 

S21: EBGFN 1, and PRFN1. EBGFN 1 - particularly at higher altitude. Between PR and PRFN is a little 
harder. In sparse terrain, it’s better to have populated areas, and FN gets lost. 

S22: EBGFN 1, and PRFN 1 . FN helps with depth perception. 

S23: EBG1, andPRl. No written comment. 

S24: EBG1, and PR1. Didn’t see any situations where he felt FN was solving a quandary for him. FN 
briefly confused him sometimes. Did not add, only distracted. 

S25: EBG1, and PR1. Don’t like FN. Confusing. Redundant. 
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S26: EBGl,andPRl. No written comments. 

S27: EBGl,andPRl. Just have a slight preference to favor not having FN. 

Summary: 

10 subjects preferred EBGFN1/PRFN1 (to not having a FN). 

14 subjects preferred EBG1/PR1 (to having a FN). 

2 subjects preferred no FN on the EBG1 texture, but a FN on the PR texture. 

1 subject had no preference. 

4. Please rate on a scale of -10 to +10 whether the use of the fishnet overlay improve situation 
awareness, terrain awareness, and pilot performance. -10 absolutely makes it worse (a definite get 
rid off); +10 absolutely makes it better (a must have). Please indicate your rationale for making that 
decision. 

SI : At high altitude, maybe a 1 or a 2 - it’s kind of a wash. At low altitudes and approaches, would go up 
to a 5 or 6. 

S2: -10. Distracting. Elands down. Except for CC, then FN is necessary, shows elevations. 

S3: Would not get rid of completely. 5. Depends on terrain and display concept. 

S4: In cruise, 5000’ AGL, FN helps out with terrain. As get closer to the airport, it’s easy to lose in the 
detail - can mistake FN with roads, etc. 

S5: 0. 

S6: -1. Doesn’t help differentiate terrain features. Not as realistic with FN. 

S7: 5. Not absolutely necessary, but good to have. 

S8: Texture dependent. EBG - -3, PR - +5, CC - +10 (a must have). 

S9: Depending on resolution, sometimes have dark areas, etc., and can aid in these cases. 

S10: 6. Adds without taking anything away. Very intuitive. 

SI 1: 5. Did like it. Did improve SA. Seemed like a subtle improvement. 

S12: 3. In general, would say 3. But, really depends on texturing concept. PR, may have a stronger need 
for FN. CC, the FN is the only terrain identification that you have. 

S13: 7. Really enhances the unit. 

S14: Texture dependent. EBG - +5; EBG has good shadowing and color contrast. That alone is enough 
to catch his eye. Adding FN is just some improvement, gives coordinate direction. PR - +10; FN helps 
you see it quicker. CC - +10; a definite, otherwise, you’d just have a standard Al. 

S15: 0. Could live with and without. If it costs more or takes up memory, then would like the money 
and memory to be used for something else. On the CC, it’s more important. 

SI 6: 0. Pretty sure it could enhance for some people. For him, it didn’t help or hurt. 

SI 7: -8. Was very distracting. Didn’t help with terrain. Didn’t like at all. 

SI 8: -1. In certain circumstances (PR, EBG), FN would depict flat terrain when coloring shows 

differently. CC - FN is a necessity to depict terrain. 

SI 9: -5. FN stinks. Most cases, added clutter and confusion. Although, had to have it for CC. 

S20: 7. Doesn’t have to be there, but does improve. Doesn’t clutter. 

S21: 4. Overall. FN doesn’t help in all circumstances. Doesn’t think needed as much for higher 
resolution. Would be +10 for CC. 

S22: 5. FN helps with depth perception. 

S23: 0. Overall. Flowever, would like to break up ratings as a function of DEM, because he thought that 
the FN helped for the DEM30, but provided little for DEM3 and DEMI. 

S24: -2. When there, a bit of a distraction. But wouldn’t throw display away if had FN. 

S25: -5. Doesn’t really mess everything up, would like to see it go, though. 

S26: 0. FN didn’t help me, but didn’t annoy or bother me, either. FN on CC does help, though, with no 
difference in color, showing terrain features can’t be picked out without FN. 
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S27: 0. Doesn’t necessarily detract (sometimes it does add clutter, and sometimes adds a little definition 
- these two together wash each other out). 

Average: 1.5. 

5. If the rating for question #7 was less than +10, would could be done, if anything, to improve the 
fishnet to increase the rating? 

SI: It’s just the nature of the FN, not a matter of improving. Since the FN is artificial, concept is 
deficient. Can’t thi nk of anyway to improve. Maybe make it on/off selectable. 

S2: Nothing. Just get rid of it. 

S3: Maybe have it pilot selectable, turn on/off. Change color, maybe a light yellow? At least play around 
with the color some. Likes that it was morphed to terrain. 

S4: Increase distance maybe to 1000’ by 1000’. And, have bigger contrast between roads and FN. 

S5: Don’t think it’s needed. EBG is that good. Maybe change the color and the grid size. 

S6: No. 

S7: Play with grid spacing - maybe smaller for steeper terrain. Also, maybe use complementary colors to 
the pixels that it’s replacing. Maybe have selectable. 

S8: For the EBG concept, FN does not enhance at all. For the PR, the FN needs to be there, and it was 
fine the way it was. 

S9: No. Maybe make it selectable. 

S10: Color contrast could be a little better. Maybe a toggle between different colors. Maybe at high 
altitude (above 5000’ AGL), could be a little less detailed (like 1000’ or 1500’ squares). 

SI 1 : Maybe on/off selectable. Maybe a little more defined. 

S12: Having it change to a contrasting color, automatically. On/Off selectable may help, however, may 
add a level of complexity. 

S13: Maybe coloring. In some instances, FN disappears, so maybe play with spacing. 

SI 4: CC - make the FN darker and a fine line. When you get close to the ground, the line gets a little 
fuzzy. Make it sharper. Extend it further into the horizon. 

SI 5: Tighten spacing - maybe increase by two points. Maybe make topography lines. 

S16: No. 

S17: No. 

SI 8: No, because of lower resolutions on EBG (FN would depict flat terrain, when coloring shows 
differently). 

SI 9: Change to wireframe like video games, start out with FN connecting and shading terrain elevation 
points instead of overlay. We’ve put in really good terrain, so just get rid of the FN. 

S20: Only useful in a short distance. Level of detail switching - would like to see further out. Pilot 
selectable would be nice. 

S21 : Maybe a color change. Way it’s presented is nice. Didn’t confuse with roads. On/off nice. 

S22: Seemed like it worked well. No confusion between roads and grid. Perhaps color for grid should be 
different than roads. 

S23: Increasing the color contrast would significantly help the discriminability of the FN, especially 
against PR. 

S24: Maybe a different coloring or changing shading. Change level of detail switching to see further. 

S25: Fly by dead-reckoning - would like FN to stand out from landmarks a little. With CC, definitely 
make FN a different color. 

S26: No written comments. 

S27: Coloration - brighten or darken. Change so FN isn’t so similar to roads, etc. Maybe tighten it up - 
at least on CC. Possibly have on/off selectable switch. 
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6. Please rank order, combining display concept and DEM, which was your preference in terms of 
most enhancing flight safety for general aviation aircraft. 

PR+FN DEM = 30 3 1 

EBG+FN DEM = 30 3 1 

CC+FN DEM = 30 1 


Pilot 

PRFN30 

PRFN3 

PRFN1 


EBGFN30 

EBGFN3 

EBGFN1 


CCFN30 

CCFN1 

1 

3 

2 

1 


3 

2 

1 


2 

1 

2 

3 

2 

1 


3 

2 

1 


2 

1 

3 

3 

2 

1 


3 

2 

1 


2 

1 

4 

3 

2 

1 


3 

2 

1 


2 

1 

5 

3 

2 

1 


3 

2 

1 


2 

1 

6 

3 

2 

1 


3 

2 

1 


2 

1 

7 

3 

2 

1 


3 

2 

1 


2 

1 

8 

3 

2 

1 


3 

2 

1 


2 

1 

9 

3 

2 

1 


3 

2 

1 


2 

1 

10 

3 

2 

1 


3 

2 

1 


2 

1 

11 

3 

2 

1 


3 

2 

1 


2 

1 

12 

3 

1 

2 


3 

1 

2 


2 

1 

13 

3 

2 

1 


3 

2 

1 


2 

1 

14 

3 

2 

1 


3 

2 

1 


2 

1 

15 

3 

2 

1 


3 

2 

1 


2 

1 

16 

3 

2 

1 


3 

2 

1 


2 

1 

17 

3 

2 

1 


3 

2 

1 


1 

2 

18 

3 

2 

1 


3 

2 

1 


2 

1 

19 

3 

2 

1 


3 

2 

1 


2 

1 

20 

3 

2 

1 


3 

2 

1 


2 

1 

21 

3 

2 

1 


3 

2 

1 


2 

1 

22 

3 

2 

1 


3 

2 

1 


2 

1 

23 

3 

2 

1 


3 

2 

1 


2 

1 

24 

3 

2 

1 


3 

2 

1 


2 

1 

25 

3 

2 

1 


3 

2 

1 


2 

1 

26 

3 

2 

1 


3 

2 

1 


2 

1 

27 

3 

2 

1 


3 

2 

1 


2 

1 

Max 

3 

2 

2 


3 

2 

2 


2 

2 

Min 

3 

1 

1 


3 

1 

1 


1 

1 

Ave 

3.0 

1.0 

1.0 


3.0 

2.0 

1.0 


2.0 

1.0 

STDV 

0.0 

0.2 

0.2 


0.0 

0.2 

0.2 


0.2 

0.2 
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7. Please indicate the reason for your choices above. 


SI: 30DEM is almost misleading in all configurations. Takes terrain with contour and turns to farmland. 
Bad for engine out. DEMI over DEM30 any day, any configuration. DEMI gives more information and 
defines ridgelines. 

S2: 3 and 30 are blurry. Can’t see deviations. Can see shadows well in DEMI, but 30 is too blurry. All are 
acceptable. 

S3: DEMI - more visual cues to look at. Gives you curvature, where DEM30 is flat. 

S4: Quite a big difference between DEM3 and DEM30. Between DEMI and DEM3 is a tough call, but 
not sure how much better 1 is over 3. As a pilot, would want to have 1. However, wouldn’t pay a lot 
more for DEMI. Thi nks 3 is good, but if cost wasn’t an issue, would choose DEMI. DEM30, in general, 
if rely on to avoid terrain, gives a false sense of security. Would tend to trust DEM30 more than fidelity 
warrants. 

S5: Did not see a magnitude of difference between 30, 3, and 1. 1 and 3 definitely not much difference, 
so maybe you don’t need 1? EBG30 would be preferred over PR concept. If 30 DEM that we’re seeing is 
better than should be, then these results are off. 

S6: No written comments. 

S7: The more information you have, the better. For cruise, DEMI is very good. For low (like our rare 
event), he feels he would have CFIT with DEM30. 

S8: Saw a difference, but no much from 3 to 1, but a huge difference in 3 to 30. 

S9: Resolution was not really a factor for terrain awareness - all are beneficial. But for clarity purposes, 
would definitely have a preference. 

S10: Biggest difference was between 3 and 30. The difference between 1 and 3 was appreciated, but not 
as critical. Comfortable with 3, but would go with 1 if had the choice. 

S 1 1 : High resolution is always nicer, but it’s not critical. DEM30 sometimes got smoother over too 
much. DEM3 with FN was good. 

S12: At times, during much terrain relief, DEMI seemed a little cluttered and busy. 30 looked like a 
watercolor picture. 

S13: DEMI shows a little more detail. 1 and 3 are close enough where he could live with DEM3. 

S14: DEMI - never had more resolution than needed. Mid resolution was marginal, until got close to the 
ground, then was fuzzy. 30DEM terrain gets blurry much sooner. EBG - DEM3 acceptable (would do 
the job), 30 marginal, 1 exceptional and not to the point of being too much. CCFN - didn’t use the terrain 
the same way as the other two; DEMI is most desired, DEM30 least desired, but not significantly 
different as far as usefulness. 

S15: Low resolution - can’t really see much. DEMI is better depiction as to what’s there, 3 is close 
depiction, and 30 is blurry. But, would take DEM3. Would like EBGFN3 over PRFN1. 

SI 6: DEMI - more elevation definition. DEM3 tell you it’s there, but DEMI tells you how high. 
DEM3 is adequate for the job. 

S17: DEM3 is pretty good, until you see DEMI. 3 would be adequate, if 1 was too expensive. 30 is 
distracting, blurry, fake, washed out. For EBGFN, you lose a lot more between 1 and 3 here than on PR. 
Chose CCFN30 over CCFN1 because FN faded away, and this made a difference. 

SI 8: FN throws off synthetic elevation terrain at lower resolutions. DEM30 is not enough resolution to 
use. DEM3 would be ok if price is an issue. 

SI 9: CCFN - doesn’t give him much cues at all, so DEM doesn’t really make a difference here. EBG - 
DEM3 to DEMI, not a big difference. But 1 is ideal, especially with EBG. DEM30 is almost flat, doesn’t 
give you much. PR- didn’t feel much difference at all between 1 and 3, because of the level of detail in 
the image. 30 without FN gives you perception of depth because detail is there with the 4 meter imagery. 
S20: If he could afford it, would go for the best. Likes to see what’s out there. DEM30 is usable, 
navigating over the water. 
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S21: DEM30 could possibly be dangerous. DEM3 gives enough, with a little common sense, even 
though it’s not as good as DEMI. 

S22: Overall, preferred PRFN. FN helped. DEM resolution helped. Elighest resolution always helpful. 
Take high resolution CC regardless of texture model. 

S23: The DEM choices were motivated by the “crispness” of the concept in that acuity of the display 
features was better with DEMI versus DEM30. The difference is the rating of how much worse the 30 
versus 1 is: PR- 20% worse; EBG - 40% worse; CC - 60% worse when comparing DEM30 to DEMI. 
S24: DEM3 - probably in most cases ok. Difference in 1 and 3 just a little more objectionable in PR than 
EBG. The greater resolution, the better it looked, and the more confidence he had. PR seemed most 
sensitive to resolution chance. DEM30 just doesn’t look quite right. 

S25: DEMI is really good. Deteriorates quickly between DEMI and DEM3, and even more so between 
3 and 30. 

S26: No written comments. 

S27: The tighter the density, the more accurate. Can judge heights better with higher resolutions. DEM3 
is suitable. DEMI is preferable, but DEM3 ok. DEM30 still gives you a feel, but get spoiled by the other 
stuff. CCFN30 is better than nothing. 

8. Which of the following display concepts would you most prefer to have in your aircraft? Please 
indicate the reason for each choice. 

CCFN30 OR PRFN30 

S 1 : PRFN30 - there is some elevation cues embedded in PR. 

S2: PRFN30 - gives some terrain height. Can tell where the mountain starts. 

S3: PRFN30 - no written comment. 

S4: PRFN30 - like texturing a little better. 30 resolution - only thing he’d get out of it is artificial 
horizon. Does not give enough resolution for terrain avoidance. 

S5: Skipped. 

S6: PRFN30 - background masked loc information on CCFN. Plus CCFN is not a realistic enough 
depiction. 

S7: PRFN30 - has more information, and the more information the better. Good for emergencies. 

S8: PRFN30 - can see the terrain better. CC doesn’t provide “squat” at this DEM. 

S9: PRFN30 - Color of terrain - was close to reality. 

S10: PRFN30 - because in an emergency situation, the PR texture would add something. CCFN30 did 
keep you focused on runway, with very few distractions. 

S 1 1 : PRFN30 - just a slight preference, though. PRFN30 does show what’s actually out there a bit 
better. CCFN30 - contrast with obstacles and shoreline is pretty good. 

S12: PRFN30 - the navigation abilities of it. Terrain shading does give some value. 

S 1 3 : PRFN 3 0 . No written c omment. 

S14: PRFN30. - Elard choice. Elave a better perspective of the terrain profile on PR. Peaks/valleys show 
up better. Proximity of terrain on horizon, they’re both about equal. 

S15: PRFN30 - if money wasn’t an issue. If money is an issue, would take CCFN30. 

SI 6: PRFN30 - Color, it’s prettier. 

SI 7: PRFN30 - Hand-down. So much better to tell you where you are and what’s ahead. 

S18: PRFN30 - good for emergencies. 

S 1 9: PRFN30 - no written comment. 

S20: PRFN30 - Depicts water better. Easier to differentiate the terrain you have. 

S21: PRFN30 - didn’t like CCFN30 at all. 

S22: PRFN30 - More realism and enhances SA. 

S23: PRFN30 - PR like VFR. 

S24: PRFN30 - Coloring and textures help. Doesn’t know if this would have helped in the rare event. 
S25: PRFN30 - More realistic, mind more at ease. 
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S26: Didn’t answer this question. 

S27: PRFN30 - Coloring, more real. 

Totals: 

26 out of 26 subjects chose PRFN30 over CCFN30. 

EBG1 OR EBGFN30 

SI : EBG1 - Even a stronger preference, can really see below. 

S2: EBG1 - Definitely. Don’t have FN distractions, have terrain, much more detailed. Better SA. Can 
compare with map and find yourself if you get lost. Good for engine out. DEM30 is flat. 

S3: EBG1 - first choice, overall. 

S4: EBG1 - Peaks much more apparent with DEMI. 

S5: Skipped. 

S6: EBG1 - Resolution differentiates higher elevations more so than in the DEM30. 

S7: EBG1 - More information the better - resolution is higher. 

S8: EBG1 - Pleads and toes above. FN distracts, here. 

S9: EBG1 - Level of detail and high resolution. 

S10: EBG1 - so good, even without FN would like this. EBGFN30 too planar and two-dimensional. 

S 1 1 : EBG1 - the high resolution is the main selling point. EBGFN30 - FN and low resolution just 
doesn’t give you enough. 

S12: EBG1 - Detailed information - DEMI versus DEM30 outweighs information from the FN. FN 
requires some interpretation. 

S13: EBG1 - Terrain is much more evident. 

S14: EBG1 - FN doesn’t quite get you there. EBG1 is definitely better. 

SI 5: EBG1 - resolution is so much better, and he doesn’t care for FN. 

S16: EBG1 - color-wise, they’re the same. But, I like more detail (higher DEM resolution) in terrain 
depiction. 

SI 7: EBG1 - No FN to distract. Resolution is more accurate. 

S 1 8: EBG1 - lot of terrain information. 

SI 9: EBG1 - no written comment. 

S20: EBG1 - higher resolution is more important than FN. 

S21 : EBG1 - Didn’t need FN here, the relief was evident. 

S22: EBG1 - Resolution is very powerful. 

S23: EBG1 - high resolution; DEM is better than FN. 

S24: EBG1 - Always choose the high resolution. 

S25: EBG1 - Contours much more visible. 

S26: Didn’t answer this question. 

S27: EBG1 - Get more robust feel to it. Picks up more of hills/valleys. More realistic. 

Totals: 26 out of 26 subjects chose EBG1 over EBGFN30. 

CCFN1 OR EBGFN30 

SI: CCFN1 - but these are very close. There are severe problems with both of them. DEM30 is 

misleading, even though prefer EBGFN over CCFN. 

S2: EBGFN30 - can see where mountain starts. Likes coloring better. 

S3: EBGFN30 - gives him more to work with, more detail between aircraft and horizon, whereas the 
CCFN1, he can see ridges and elevations at the edge of horizon, but not much up close. 

S4: EBGFN30 - Resolution does NOT make a difference, in this case. 

S5: Skipped. 

S6: EBGFN30 - with the CCFN, just can’t see as well. 
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S7: EBGFN30 - can see different terrain a little further. CCFN1 gives you more information, but is 
harder to see. 

S8: EBGFN30 - Colors on the EBG, can do a lot with as a mechanism for deterrent. 

S9: EBGFN30 - likes the terrain depiction better. 

S10: EBGFN30 - still provided enough information. Just likes the way EBG works - it’s very intuitive. 
SI 1: CCFN1 - slight preference. Man-made obstacles are a lot more obvious. 

S12: EBGFN30 - elevation color coding gives more intuitive information than interpreting the FN. 

S13: EBGFN30 - Coloring definitely helps here. CCFNl is almost an AI. 

S14: EBGFN30 - Can inteipret the terrain a little faster with EBG. SA of what’s going on is better. 

S15: EBGFN30 - with EBG, even with the blurriness, colors help with depiction. 

SI 6: EBGFN30 - CC, really hard for him to interpret how high and where mountain is. With EBG, 
coloring helps mountains stand out. 

SI 7: EBGFN30 - Prefer coloring scheme. 

S18: CCFNl - Wants to stay away from FN on EBG30, because at that resolution, FN is deceiving. 
CCFN gives some terrain information. 

SI 9: EBGFN30 - resolution isn’t enough in this comparison, would rather have shading. 

S20: EBGFN30 - likes these colors better. Probably would go with this, but if he had time to see how 
CC works, might get used to it. For CC, the FN is very important. At the DEMI, the CCFN is more 
detailed, but harder for him to inteipret. 

S21 : EBGFN30 - even though resolution is lower, the shading gives much better picture of relief. 

S22: CCFNl - slight edge. It’s a touch choice, though. Likes the clarity of higher DEM. Helps for down- 
low operations. 

S23: EBGFN30 - although it’s close. 

S24: EBGFN30 - CC is the exception to the high resolution rule. Even with DEMI, doesn’t feel he gets 
information as quick as with other displays. 

S25: EBGFN30 - The more realistic things are, the more peace of mind you have. If the FN were 
implemented better, might have chosen the CCFN 1 . 

S26: Didn’t answer this question. 

S27: EBGFN30 - Less dense, but shading gives good information. 

Totals: 

4 out of 26 subjects chose CCFNl over EBGFN30. 

22 out of 26 subjects chose EBGFN30 over CCFNl. 
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9. (Combined questions 9 and 10 on final questionnaire) If you had to select between two different 
FOVs that may be pilot selectable, which two FOVs would be chosen and why? What were the 
reasons for the choices of FOVs, for the different phases of flight? 



FOVs 




Pilot 

22 

30 

60 

90 

1 





2 





3 





4 





5 





6 





7 





8 





9 





10 





11 





12 





13 





14 





15 





16 





17 





18 





19 





20 





21 





22 





23 





24 





25 





26 





27 





Totals 

13 

8 

25 

8 


SI: 30, 60. 90 is too compress, 22 is too sensitive. 60 can be used for up and away, and turns. 30 is good 
for close in finals, because needs sensitivity to follow GS and Loc - want to be able to see about !4° of 
movement. Sensitivity and control are two issues of concerns, when it comes to FOV selection. 

S2: 90, 60. 90 - with tunnels, as far as coming into approaches. 60 - gives a lot of terrain information. 
Opens symbology to get a little more precise. In turns, needed the expanded FOVs to see where he’s 
going. 

S3: 22, 90. 90 - good for flying, airwork. When making turns, it’s easier. Seeing what was ahead of him 
helped him make decision on what was coming. Couldn’t pick up runway, though. 22 - on approach, 
maybe at Outer marker. GS line is easier to use, also, at 22. Still had trouble transitioning from 90 to 22. 
S4: 90, 60. 90 good on approach segments where precision is needed. 60 better for cruise. During cruise 
with boxes, less boxes are desired. Once below 400’ AGL, preferred 30, and at last 100’ went to 30 to 
see the runway better. 
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S5: 60, hands down across the board. Prefer 30 and 90 equally, but will choose 90. 22 has no benefit. 60 
seems to most closely represent OTW. 22 and 30 did not provide information below him. 90 in turns 
helped. Strongly opinionated against 22 - misrepresented what’s out there, thinks it’s misleading, 
missing information too close in front of you. 

S6: 60, 90. 90, you have more view under the nose. 

S7: 60, 22. Likes 60 in cruise. Would give up 90 on approach for 60 in cruise. If had more boxes per 
mile with 60, would really like 60. Maybe we could decouple box spacing so FOV is selectable and 
number of boxes is also selectable. Feels that 22 is almost absolutely needed for approach. 

S8: 60, 30. 30 was best for approach, he could really see aiming point. 60 - gave him a better view for 
the turns. 90 overcompensated. Prefer to answer not maneuver independent. 

S9: 60, 22. 60 for en route, tunnels. Symbology not as sensitive, more controllable. 22 for approach, 
near runway. Gives a real-time view. 

S10: 60, 22. 60 gives good balance between 90 and 30, for en route. 22 had awesome precision for 
flying on final. All choices had their own applications. Used 30 the least. 90, controls were the least 
sensitive, and SA was the broadest. 

SI 1: 60, 22. 22 - really nice on short final. 60 - best compromise (as far as not having 90), and keeps 
sensitivity level down on approach. 

S12: 60, 30. 60 - more for en route wide FOV situations. Terrain Awareness. 30 - more for when closer, 
lower level maneuvering approaches, finer control. 22 - too squirrelly; chases VV. Likes 90 from a 
controllability standpoint. 

S13: 60, 22. 60 gives good FOV without having sensitivity. Wants 22 when close in at runway. At 
higher altitude, really liked the 90 to see as much all around as possible. 90 gives a broader view. 

S14: 60, 22. 22 - likes it on short approach; got good perspective of runway and navigation guidance 
(VV) on runway; terrain perspective close to runway was better. 60 - good compromise for wider angle 
for terrain and skyway boxes are easy to navigate. 

SI 5: 60,22. 22 - more realistic. Sets the base. Likes on approach and when weather is bad. 60 -to set 
up for approach and for straight and level flying. Boxes come at a good speed for a 172. 

SI 6: 60, 90. 60 - accurate amount of perception of terrain environment. 90 - have more perception 
around him and is easier to make correction when off altitude. 

S17: 30,90. 30 - for approaches; 90 - like being able to see below the aircraft. Higher up, seemed like 
90 gave her more shaipness. When lower, felt like she knew more of what was around her with FOV60. 
S18: 22,60. 22 - very precise when holding straight and level; 3° line works well. 60 - en route gives 
wider FOV, not as sensitive so easier to keep numbers; gives heading information on horizon line. 

S19: 60,90. With this symbology set and the boxes, need 60 and 90 in the turns. 30 and 22 - not very 
useful at all - lose heading information on the horizon line; don’t have flight information cues; lose your 
terrain awareness. 

S20: 22, 60. 22 - at outer marker, gave him runway view. 60 - good for en route and initial approach, 
before outer marker. Could see more as he goes. Wider FOVs help in turns. 

S21: 22, 60. 22 - on short final. 60 - en route flying, better idea of what’s around. Works for boxes on 
approach. On approaches, liked 90 and 60, but would choose 60 if had to limit. 

S22: 22, 60. 22 - helped with being placed where I wanted to be. Useful for en route. 60 - better for 
approach. 90 was not needed - too far out. 60 - giving enough terrain information for landmarks and 
runway. 

S23: 30, 60. 30 - seeing details and making adjustments (micro); and 60 - provides the best contrast for 
seeing the entire field of view. In general, 60 was chosen overall. Thought that 60 was best for 
maneuvering probably because the display is so small and need to see more in view. Larger display could 
allow a smaller FOV to be chosen. Also, interestingly, had concerns about finer grade adjustments on 
approach with the 60FOV. Therefore, qualification that smaller FOV would be optimal, although flew 60 
most of the time. 
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S24: 30, 60. 30 - descents during en route and approach, higher resolution on the pitch ladder. 60 - used 
most during en route, corresponded best with OTW. 22, showed just a little sky and a little sliver of 
terrain en route and resolution was higher on symbology than he could hold. Liked 90 for approaches 
only for more boxes in tunnel, not for field of view of terrain. 

S25: 30,60. Likes 60 for all cases. Very comfortable with 60. For peace of mind, likes to see below the 
nose. 22 - shows nothing below the aircraft. 90 - too far away. 

S26: 22, 60. 60 - most realistic look, depth perception. 22 - on approach, close to runway, gives better 
look at runway and aiming point. 

S27: 30, 60. Have to go with compromise. 22 - things happened to fast. 30 - gives tighter, more 
control, and still had a good view of terrain. 60 - good compromise, only slightly prefer over 90. In 
general, likes 90 for tunnels, 60 for overview mode, and 30 for precision. 


11. You chose, in the block questionnaire, a different FOV for the “x” display concept. What was 
the reason for that choice? (The intent of this question is to relate back to the display concept 
selection the subject made for question 4/5 of the block questionnaire.) 

SI: N/A 

S2: CCFN and FN in general. Would use 22 and 30 in these cases, so he could zero in on instruments. 
Coloring and FN were so distracting that he wanted terrain to go away. 

S3: Same FOV for all display concepts. 

S4: N/A 
S5: N/A 
S6: N/A 

S7: Used FOV30 for the high resolution EBG concepts, at low altitudes. The EBG gives you elevation 
cues, enabling the closer FOV for symbology. With the PR, you don’t have many cues to elevation, it’s 
more about vegetation. 

S8: N/A 
S9: N/A 

S10: Used only 90 in CCFNs - could focus on the broad settings with this texture. 

S 1 1 : For the baseline SVS display, the symbology was easier to interpret. So using 30FOV helped him to 
hold heading a little tighter. 

S12: CC, because he felt like he had to zoom in to be able to see and interpret FN. 

S13: N/A 

S14: With PR and EBG, it didn’t make a difference. With CCFN, would go to 22 sooner on approach (at 
least at final approach fix). Maybe because runway stands out and not much terrain to look at with CCFN. 
S15: N/A. 

SI 6: N/A. 

SI 7: For the low altitude runs, typically 60FOV was used. However, for EBG, 90 was preferred - just 
seemed to feel better. Maybe seeing more of it gave her a better idea of where she was. 

SI 8: Terrain did not make a difference on CCFN - used 22 because more precise flying. 

SI 9: N/A. 

S20: Might use 90 and CCFN to get a better feel for what’s further out. 

S21: Never. 

S22: N/A. 

S23: N/A. 

S24: N/A. 

S25: N/A. 

S26: N/A. 

S27: PR1 - have less of a need for tighter FOV, because the resolution is so good. Could more 
accurately fly with PR1. 
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12. You were flying display condition when you experienced that your aircraft was 

significantly below the indicated altitude (the anomaly). Did the display concept allow you to 
determine where your aircraft was in relation to terrain? 

SI : CCFN30. Yes, the first thing he noticed was terrain above the horizon. Would not have picked up on 
that with conventional displays. Feels he would have picked it up faster with better, more detailed display. 
S2: PRFN3. Started to show up on PFD, but he thi nks without FN and with DEMI, would have noticed 
sooner. Really started to see when he looked OTW. The PFD prompted him to look OTW to verify terrain 
was getting closer. With this view, probably would have waited longer to pull up. 

S3: EBGFN30. Yes. 

S4: EBG1. Yes. 

S5: CCFN30. If he would have had EBG and PR, he feels he would have seen terrain above horizon 
more quickly. But he did see it in time with CCFN30 to avoid terrain. 

S6: PRFN1. Yes. 

S7: EBGFN3. Yes. 

S8: EBGFN1. Absolutely. Made him look at the MX20. 

S9: PRFN30. Yes. 

S10: PRl.Yes. 

SI 1 : CCFN30. Yes. Noticed terrain on PFD, and was trying to figure out why it was so close when it 
wasn’t before. 

S12: EBGl.Yes. 

S13: EBGFN1. Definitely. 

S14: PR1. Yes, did a good job. 

S15: PRFN1. Yes. 

S16: EBGFN3. Yes. 

SI 7. PRFN3. Yes - very close to OTW. 

S18: EBGFN30. Yes. 

SI 9: PRFN30. Yes. 

S20: CCFN1. On a limited basis. 

S21: CCFN30. No. 

S22: EBG1. Potential simulator problems clouding event. Didn’t panic - thought it was a sim problem. 
S23: EBGFN1. Yes. 

S24: PR1. Yes. The only thing - no depth perception (stereoscopic). Would like to know how far away 
things are. 

S25: PRFN1. Had a feeling from SVS that he was ok with relationship to the terrain. Looking OTW 
was what told him something was wrong. 

S26: EBGFN3. Yes. 

S27: PR3 (because PRFN3 was not working properly). Definitely. DEM3 versus DEMI wasn’t’ really 
pronounced until you were down on the deck. But DEM3 allowed him to see what he needed. 

13. What display features were most significant in terrain avoidance (if the subject did not detect 
the anomaly, ask them was features would be most significant generally for terrain avoidance for 
that display concept)? 

SI: Terrain above the horizon. 

S2: Shadows, mountain coming up at you, horizon above you. 

S3: Something didn’t feel right. Felt he was closer to ground than should be. Terrain and towers - he felt 
like he was going to hit. 

S4: Was able to tell very clearly that he was below the terrain. Was in simulator, however, and was not 
worried about crashing - thought we wanted to keep our data, so stuck to his numbers. 
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S5: Started doubting himself until he saw the OTW ground. Thought maybe he was interpreting 

CCFN30 wrong. He was looking for goal posts. 

S6: Towers were tell-tale features - it appeared to him that he was at or below the towers. 

S7 : When getting close to hill, the horizon line was on top of and below the hill. Knew that wasn’t right. 
Had forgotten about the MX20. Then looked OTW to confirm. 

S8: The lighter areas were too close. 

S9: VV was below the terrain. Saw OTW equaling inside, and it was large and scary. 

S10: Terrain depiction itself (amount of green in picture). He changed FOV to 22 30 seconds before he 
said anything. At 22, saw a lot of ground (green), whereas in the runs before this, wouldn’t see much 
ground. 

SI 1: CCFN30 doesn’t really have good depth perception. If he would have been flying a PR texture in 
this scenario, he feels he would have caught it sooner. 

SI 2: Noticed towers above elevation, then ground above elevation. Checked MX20 to see, and struggled 
to figure out which one to believe. Then looked OTW and saw tower. 

S13: Hill and tower being above him. Did not use the MX20. Did look OTW to verify how low he was. 
SI 4: Proximity of ridge line in lower center of display gave sensation that he was really close to ground. 
Then he saw terrain was above the horizon line. Obstacles really stood out. 

S15: Did not believe it. Simulation makes him feel higher than he really was. Trying to hold altitude, 
but was in a sim, so didn’t think it was a big deal. 

SI 6: Realized that he was in valley, and saw how close mountain was (at eye-level). Saw towers. He 
thinks that texture made him look closer than he really was. Then, looked OTW for verification. 

S17: Could see mountain tops and towers at 5380’, and there’s no way she can descend another 300’. 
Did not look at the MX20 at all. 

SI 8: Didn’t speak up soon enough. Saw antenna on display and knew ridge line was becoming above 
him. Noticed it at 5900’, but didn’t say anything - wanted to believe his numbers. 

SI 9: Didn’t look at MX20. Noticed that horizon was higher than normal. Picture just looked like he was 
a lot lower than yesterday. Compelling enough to not go lower (at 5350’). Typically, terrain on a display 
is low priority, because of validation. So, you are more inclined to fly gauges because that’s what you are 
trained to do, and because they are certified to higher accuracy. Very good discussion about this - look at 
tape. 

S20: Primary cue was OTW. Without that visual cue OTW, it might be a different story. If he had EBG, 
he might have picked it up sooner. 

S21: Visibility set to one mile OTW. Noticed OTW, something was amiss. Looked for something on 
display and never found it. CCFN1 would NOT have been any better. 

S22: Would have preferred PR and PRFN for better terrain awareness. Tough call between PR and EBG. 
Natural eye would prefer PR. 

S23: Didn’t see any failure of altimeter or realize impending danger until saw OTW of towers. Issue was 
judging how far the terrain was even though the VV was trained right on the mountain peak. At the last 
moment, after judging OTW, made a left turn and 100’ increase in altitude maneuver to avoid terrain. 

S24: Looked head-down - could tell difference from what he saw before. Prompted him to look up 
(OTW) to verify. Don’t think the CCFNs would have clued him in as quickly. 

S25: Saw OTW, and this was the only cue. Would have noted sooner if he were flying the real thing. 
Head down - just didn’t feel like he was that close to the terrain. 

S26: Terrain on display looked like I was barely passing over it, could detect ridge tops easily. 

S27: Feel of being down in terrain. Didn’t look at MX20. OTW didn’t cue him. 
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14. Please indicate your level of confidence in separation from obstacles (e.g., towers) that you 
evaluate a pilot would experience for each display concept (on a 1 to 7 scale). 


Pilot 

Baseline 

CCFN 1 

CCFN30 

EBG1 

EBGFN1 

EBGFN3 

EBGFN30 

PR1 

PRFN1 

PRFN3 

PRFN30 

1 

3 

5 

5 

6 

6 

6 

6 

4 

4 

4 

4 

2 

1 

3 

2 

6 

5 

4 

4 

6 

5 

4 

4 

3 












4 

4 

5 

5 

7 

7 

7 

7 

6 

6 

6 

6 

5 

1 

6 

6 

5 

5 

5 

5 

4 

4 

4 

4 

6 

2 

3 

3 

6 

5 

4 

3.5 

5.5 

5 

4 

4 

7 

1 



6 

6 

6 

6 

6 

6 

6 

6 

8 

1 

6 

6 

4 

4 

4 

4 

3 

3 

3 

3 

9 

3 

4 

4 

6 

6 

6 

5 

7 

6 

6 

5 

10 

3 

5 

5 

6 

6 

6 

5 

6 

6 

5 

5 

11 

1 

4 

5.5 

5 

5.5 

5 

4 

6.5 

7 

6 

4.5 

12 

1 

6 

6 

5 

5 

5 

5 

3 

3 

3 

4 

13 

1 

3 

3 

5 

5 

5 

5 

6 

6 

6 

6 

14 

3 

3 

3 

5 

6 

5 

4 

5 

6 

5 

4 

15 

1 

4 

3 

6 

6 

6 

6 

6 

6 

6 

6 

16 

1 

2 

2 

5 

5 

5 

5 

7 

7 

7 

7 

17 

1 

1 

1 

6 

6 

6 

4 

7 

7 

7 

5 

18 

1 

6 

4 

5 

5 

4 

3 

6 

6 

4 

3 

19 

1 

6 

6 

4 

4 

4 

4 

5 

5 

5 

5 

20 

1 

3.5 

2 

6 

6 

5 

4 

5.5 

6 

4 

3.5 

21 

1 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

22 

1 

6 

4 

6 

7 

5 

5 

7 

7 

6 

5 

23 

1 

3 

2 

6 

6 

5 

4.5 

7 

7 

7 

6 

24 

1 

7 

6 

7 

7 

7 

6 

7 

7 

7 

6 

25 

1 

4 

4 

6 

6 

6 

6 

7 

7 

7 

7 

26 

1 

4 

3 

6 

6 

5 

3 

6 

6 

5 

3 

27 

1 

3 

2 

4 

4 

3 

3 

6 

6 

5 

4 

Max 

4 

7 

6 

7 

7 

7 

7 

7 

7 

7 

7 

Min 

1 

1 

1 

4 

4 

3 

3 

3 

3 

3 

3 

Ave 

1.5 

4.3 

3.9 

5.5 

5.6 

5.2 

4.7 

5.8 

5.7 

5.3 

4.8 

STDV 

0.9 

1.5 

1.6 

0.8 

0.9 

1.0 

1.1 

1.2 

1.2 

1.3 

1.2 


SI: Did not notice towers as much on PR concepts. Too much detail to have the towers stand out. EBG 
and CCFN have so little detail that the towers stand out. 

S2: BSBGBL - used MX20. CCFN1 - hard to see due to FN, hard to tell your separation from them. 
CCFN30 - some. EBG1 - colors of towers could be brighter. EBGFN1 - FN distracts from obstacles. 
EBGFN3 - because of FN and resolution, hard to find because focusing on blurry screen. EBGFN30 - 
same as EBGFN3, but more blurry. PR1 - due to color of towers, didn’t stand out well until right on top. 
PRFN1 - FN distracts. PRFN3 and PRFN30 - can still see towers, but have to focus. 

S3 : Things are starting to run together, and doesn’t remember. So, left this one blank. 
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S4: Felt he could pick up towers from a greater distance with all EBG concepts. For PR concepts, darker 
colors seemed to hide the towers. Flowever, both PR and EBG are acceptable. Used the MX20 for the 
standard gauges case. 

S5: Could see the towers in the PR concepts, but the towers didn’t stand out nearly as well as with the 
EBG concepts, or the CCFN concepts. 

S6: Resolution doesn’t matter between CCFNl and CCFN30. EBGFN1 - FN detracted a little bit. 
EBGFN3 - resolution does matter, here. PRFN1 - FN detracted. PRFN3 and PRFN30 - resolution does 
NOT matter, here. 

S7: Answered for all except the CCFN concepts, because he didn’t remember. 

S8: Towers didn’t show up as well on EBG concepts as they did on CCFNs. For the PRs, he couldn’t 
recall what the towers looked like, so figured his rating should be poor. In general, FN may have reduced 
obstacle appearance. 

S9: With the 30DEMs, you start to lose a little clarity, but still nice to have. 

S10: Answered these with the assumption of using MX20, too. 

SI 1 : The more you fly this route, the more you know what to look for. BSBG - gave no information. 
CCFNl - not a real sharp, good feeling, but does give information. CCFN30 - high contrast level. EBG1 
- A little harder to pick out towers. EBGFN1 - helped with a bit of overlay on terrain. EBGFN3 - lose 
some of the contours, just a little harder to pick out towers. EBGFN30 - no written comment. PR1 - can 
see towns, towers. PRFN1 - no written comment. PRFN3 - losing some resolution, but could pick out 
towers. PRFN30 - does know that there is a city, but now smoothed. 

S12: The one benefit of CCFN - could see towers very well. EBGs - obstacles stood out reasonably 
well, with color being relatively constant. For PRs - obstacles tended to get lost in background, until you 
got on top of them. However, the colors were washed out enough on the PRFN30 to make the towers 
stand out more. 

S 1 3 : No written c omments . 

S14: Answered this with the assumption that you are flying VFR. For the CCFNs, the towers showed up 
really well, but couldn’t decipher where they were in relation to ownship. Adding FN to EBG1, enhances 
terrain profile and gives a little more perspective. PR1 - still pretty compelling, and very comfortable 
with this display. 

S 1 5 : Resolution does matter on towers for CCFN. For all EBGs and PRs, ranked them the same, because 
color is the main factor - with the coloring, could see towers very well. 

SI 6: CCFNl - vague. CCFN30 - resolution doesn’t matter, here. Ranked all EBGs the same, because 
resolution and FN doesn’t matter. Ranked PRs all the same - towers really stood out. 

SI 7: BRD - no tower information. CCFNl and CCFN30 - don’t remember seeing towers on this one. 
EBG1 - pretty good. EBGFN1 - FN didn’t distract in this case. EBGFN3 - enough resolution that could 
spot it well. EBGFN30 - blurriness and things running together. PR1 - prefect. PRFN1 - FN didn’t take 
away from towers. PRFN3 - resolution didn’t detract. PRFN30 - blurriness. 

S18: Obstacles really stuck out on CCFNl. A little less-so on CCFN30, because resolution does matter. 
For EBG, coloration of tower hidden a little on background (more so than PR). 

SI 9: This is independent of terrain. It’s a contrast issue - maybe could make towers yellow or something 
brighter? 

S20: CCFNl - could see towers, but could not tell where he was. In general, as resolutions go down, it 
impacts how he sees towers. 

S21 : Towers appeared the same on all for him - texturing or resolution doesn’t matter. 

S22: No written comment. 

S23: TRUST was significant factor in determining level of confidence ratings. 

S24: The only thing that CCFNl had going for it was the towers. For EBG1, EBFN1, and EBGFN3, FN 
didn’t make a difference, and he felt confident in all of them. EBGFN30 and PRFN30 - little more 
trepidation because he knew database was low resolution. 

S25 : Resolution doesn’t change ratings, and neither did FN. 
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S26: No written comments. 

S27: For the CCFNs, they are a big advantage over BRD. You could see the towers, but couldn’t tell 
how far. For the EBGs, towers were a little harder to see, but had better feel for height. 


15, 16, and 17. Please rank order the display concepts you would prefer to have in your aircraft for 
making an approach, en route cruise, dealing with an emergency situations. 



Approach 

En Route 

Emergency 

Rank 

Actual 

Mean 

Display 

Concept 

Actual 

Mean 

Display 

Concept 

Actual 

Mean 

Display 

Concept 

1 

2.7 

EBG1 

2.6 

PR1 

2.4 

PR1 

2 

2.7 

PR1 

2.6 

EBG1 

2.9 

PRFN1 

3 

3.2 

PRFN1 

3.0 

PRFN1 

2.9 

EBG1 

4 

3.3 

EBGFN1 

3.4 

EBGFN1 

3.5 

EBGFN1 

5 

5.0 

EBGFN3 

5.1 

EBGFN3 

4.9 

PRFN3 

6 

5.2 

PRFN3 

5.2 

PRFN3 

5.2 

EBGFN3 

7 

7.2 

EBGFN30 

7.1 

PRFN30 

7.1 

PRFN30 

8 

7.3 

PRFN30 

7.3 

EBGFN30 

7.3 

EBGFN30 

9 

8.6 

CCFN1 

8.7 

CCFN1 

8.9 

CCFN1 

10 

10.0 

CCFN30 

10.0 

CCFN30 

10.0 

CCFN30 

11 

11.0 

Baselines 

11.0 

Baselines 

11.0 

Baselines 


SI: No written comment. 

S2: No written comment. 

S3: No written comment. 

S4: No written comment. 

S5: Approach was dominated by loc/GS. Displays were less important during approach. Likes the 
peripheral of EBG. Terrain is more important during en route tasks. FN/no FN - has no bearing. 
DEM1/DEM3 - no bearing. Flowever, it does come into play when drop to 30. 

S6: No written comment. 

S7: When ranking all concepts, with respect to different maneuvers, talked about the fact that he didn’t 
know how realistic the photos for PR are - and if he knew that the fields were not overgrown, his answers 
would change. But, he doesn’t know. Also, in the rankings, he felt that all EBG concepts were very close, 
and all PR concepts were very close - that FN or resolution didn’t matter that much when it comes to 
general preference. 

S8: No written comment. 

S9: No written comment. 

S10: PR concepts are really helpful in situations where it’s good to know whether or not you are over a 
field or a suburb (i.e., emergencies). Even things rated low would be a dramatic improvement over 
current gauges. 

SI 1 : PR was easiest to interpret as far as roads go. Engine out and forcing a landing, will be very 
interested in terrain contours. 

S12: FN less important for en route. PR becomes more important than FN for determining landmarks. 
EBG - immediately obvious where low ground is, and that you should “fly to the green”. 

S 1 3 : No written c omment. 
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S14: Would pick EBGFN30 over PRFN3, because coloring on the EBG is more compelling than the PR. 
Greens and darks against a blue sky isn’t a strong contrast. 

Si 5: PR is more showy than it is functional. Feels that PRFN30 is a little blurry, and CCFN1 gives him 
more detail than the PRFN30. Fikes EBG much better than PR. 

S16: PRFN30 ranked high (above EBGFN’s for an approach) because it’s pretty. However, would like 
DEMI over DEM30 for an emergency, so would then prefer EBGFN1, EBGFN3 over PRFN30. 

SI 7: Terrain on EBG better, so for the en route (as opposed to approach) can judge better and gives more 
detail higher up. Would help in an engine out. Would, however, aim for FN thinking that it’s a road. 

SI 8: For approach, DEM is more important than coloring (so picked CCFN1 over EBGFN3). EBG3 and 
EBG30, colors distinguish altitude, that’s why chose EBG over PR for DEM 3 and 30. For en route, 
instead of CCFN1 ranked 5, he ranked it 9, because: en route, the lower resolution EBG and PR gives 
more 3-D terrain information than CCFN1. 

SI 9: For approach, PR imagery helped a lot. He didn’t expect this, so PRFN30 is preferred over 
EBGFN3. For en route, EBG was better at higher altitudes, because the green of the PRs just blurred/ran 
together. Once you get to DEM30, PR works better, because having the 4m imagery is better than the 
flatness. 

S20: For approach, prefer high resolution, but at constant DEM, likes EBG now over PR. For en route, in 
unfamiliar terrain, EBG is better. Might change in Alaska - in familiar terrain, PR might be better. Might 
be interesting to see different things on a non-instrument approach. In an emergency situation, PR might 
give him better information. 

S21: For approach, bumps in ground easier to pick up on slightly less detailed terrain (EBG). However, 
would switch for the en route, because feels that at higher altitude, easier to pick out relief on PR than 
EBG, atDEM3. 

S22: No written comments. 

S23: These ratings are identical regardless of condition. Again, confidence and trust were the significant 
factors here. 

S24: In approach phase, ranked CCFN1 5, because it does show him towers, runways, etc. As long as 
he’s on his approach path, should have to worry with terrain. For the low resolution (DEM30) elevation 
data, PR looks more convoluted than EBG. For en route, might change the CCFN rating - move it 
towards the back. 

S25: FN confuses him faster on EBG, so would like to stay with PR longer for all of the rankings. 

S26: No written comment. 

S27 : With the standard gauges, you lose so much navigation information. Likes the coloring in the PR. 
PR also gives vegetation information. 


18. Please provide what you consider the best features of each of the following display concepts: 
CCFN1 

S 1 : Obstacle and runway portrayal. 

S2: Instruments, obstacles. 

S3 : Differential and color. FN helps identify contour. 

S4: Good for artificial horizon, good for emergencies (no spin). 

S5: Good obstacle contrast. Will tell you that you’ll fly into the ground. 

S6: FN (different color terrain may make a difference). 

S7 : FN - allows to see terrain. 

S8: FN, symbology. 

S9: Provides terrain in IMG. 

S10: Allows you to focus on wherever/whatever you’re up to. 

SI 1 : Fairly decent idea of terrain; FN; sharper contrast about everything. 

S12: Avoids cluttering background; obstacles stand out. 
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S13: None. 

S14: Visual of obstacles and runway. 

S15: FN. 

S16: No. Can see towers. 

S 1 7: Little bit of slope on the horizon. 

S18: Tells where ground is, contours. 

SI 9: Obstacles stand out, better than nothing. 

S20: Can see some terrain. 

S2 1 : Better than nothing. 

S22: FN and better clarity. 

S23 : Simple, easy to interpret. 

S24: Towers, runways easy to pick out. 

S25: Beats hands down standard gauges. 

S26: FN - it’s the only way to pick out terrain. 

S27 : Better than nothing. Obstacles. 

CCFN30 

S 1 : Obstacle and runway portrayal. 

S2: Obstacles, gauges. 

S3: Likes DEMI better, FN helps ID contour. 

S4: Good for artificial horizon, good for emergencies (no spin). 

S5: Good obstacle contrast. Will tell you that you’ll fly into the ground. 

S6: Maybe the FN. 

S7 : FN - allows to see terrain. 

S8: Symbology. 

S9: Provides terrain in IMC. 

S10: Not much good, but would take over standard gauges. 

SI 1 : Some useful information. Fairly decent idea of terrain; FN; sharper contrast about everything. 

S12: Obstacles; reduces clutter. 

S13: None. 

S14: Visual of obstacles and runway. 

S15: Not much. 

S16: No. 

S 1 7: Little bit of slope on the horizon. 

S 1 8 : No written c omment. 

SI 9: Gives you motion, that’s it - faster when lower. 

S20: No. 

S2 1 : Better than gauges. 

S22: More useful than basic 7 and BSBG. A lot of information on screen. 

S23 : Simple, easy to interpret. 

S24: Towers, runways easy to pick out. 

S25: Beats hands down standard gauges. 

S26: FN - it’s the only way to pick out terrain. 

S27: Obstacles. 

EBG1 

SI : Shadow cues, lack of distractions. Gives him information that he wants. 

S2: Terrain avoidance, emergency scenarios. 

S3: Sharper than the rest; good contrast between sky and ground, and within ground; very friendly. 

S4: If money weren’t an object, would choose this. Like the graduated color scheme - lighter colors 
towards the top. 
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S5: Great terrain portrayal. Good SA. Nice to look at. 

S6: Color levels and shading of differential elevations. 

S7: High resolution. 

S8: Likes everything, favorite, symbology. 

S9: Provides terrain in IMC. Likes topographical representation. 

S10: Excellent lay of the land, very intuitive. 

S 1 1 : See lay of the land pretty well. 

S12: A lot of detail of terrain features; elevations become intuitive. 

S 1 3 : At higher altitude, had good definition. 

S14: Terrain change in elevation close and at horizon. 

S15: Love it. Sharpness. Can see valleys. Color variation. 

SI 6: Everything is good. 

SI 7: So detailed and nice. Have a good idea of what she’s flying over. 

SI 8: Awesome. A lot of good contour information. 

SI 9: Shows terrain gradient, find low spots easily. 

S20: Very easy to read, tell elevation. 

S21: Likes shadowing and shading. Easy to see things in the distance. 

S22: Resolution realistic depiction. 

S23: Very crisp and gave good details of surface. 

S24: Contours immediately obvious. 

S25: Excellent. Lots of information. 

S26: Color/shading - defines elevations well. 

S27 : Good feel for terrain, intuitive. 

EBGFN1 

SI: Shadow cues, lack of distractions. Gives him information that he wants. FN helps at low altitudes. 

S2: Terrain avoidance, emergency scenarios. 

S3 : Sharper than the rest; good contrast between sky and ground, and within ground; very friendly. FN 
was not a distracter. Did help. Only would use it at certain times - would like selectable. 

S4: If money weren’t an object, would choose this. Like the graduated color scheme - lighter colors 
towards the top. 

S5: Great terrain portrayal. Good SA. Nice to look at. 

S6: Color levels and shading of differential elevations. 

S7 : Gives him everything he needs to know. 

S8: Symbology. 

S9: Provides terrain in IMC. 

S10: Favorite. Excellent lay of the land, very intuitive. 

S 1 1 : FN helps a little bit. 

S12: A lot of detail of terrain features; elevations become intuitive. 

S 1 3 : Little better definition with FN. 

SI 4: Terrain change in elevation and visual proximity to terrain both in close and at horizon. 

SI 5: Love it. Sharpness. Can see valleys. Color variation. 

SI 6: Everything is good. 

SI 7: So detailed and nice. Have a good idea of what she’s flying over. 

SI 8: Awesome. A lot of good contour information. 

SI 9: Shows terrain gradient, find low spots easily. 

S20: Very easy to read, tell elevation. FN enhances depiction and distance judging. 

S2 1 : Likes shadowing and shading. Easy to see things in the distance, but a little more useful up high, 
especially in flatter areas. 

S22: RN and resolution. 

S23: Very crisp and gave good details of surface. FN added little. 


116 



S24: Contours immediately obvious. 

S25: Excellent. Lots of information. 

S26: Color/shading - defines elevations well. 

S27 : Good feel for terrain, intuitive. FN does help a little. 

EBGFN3 

SI: Shadow cues, lack of distractions. Gives him information that he wants. FN helps at low altitudes. 
Difference between DEM3 and DEMI were subtle in all textures. 

S2: Terrain avoidance, emergency scenarios. 

S3 : Sharper than the rest; good contrast between sky and ground, and within ground; very friendly. FN 
was not a distracter. Did help. Only would use it at certain times - would like selectable. 

S4: Still very effective, even with medium resolution. Like the graduated color scheme - lighter colors 
towards the top. 

S5: Great terrain portrayal. Good SA. Nice to look at. 

S6: No best feature. 

S7 : Gives him everything he needs to know. 

S8: Fine with FN; symbology. 

S9: Provides terrain in IMG. 

S10: Excellent lay of the land, very intuitive. 

S 1 1 : Still works. 

S12: Less cluttered than DEMls. Good balance of detail versus amount of information. 

S 1 3 : No written c omment. 

SI 4: Terrain change in elevation and visual proximity to terrain both in close and at horizon. 

S15: Coloring. 

SI 6: Terrain is visible, but not as visible. 

SI 7: Terrain still ok, just not as nice as DEMI. 

SI 8: Positive colors, would like without FN. 

S19: Maybe FN helps, but not too much. 

S20: Can see terrain. 

S2 1 : Texturing still pretty good. 

S22: FN still useful and realistic depiction. 

S23: Very crisp and gave good details of surface. FN added little. No change as a function of DEM. 

S24: Contours immediately obvious. 

S25: Florizon. Still have lots of information. 

S26: Color/shading - defines elevations well. 

S27 : Suitable interpretation. FN is a plus and a minus. 

EBGFN30 

SI: Color coding did help a little with elevation cues. FN not particularly useful here. Perceive that data 
resolution is higher than it actually is. 

S2: Terrain avoidance, emergency scenarios. 

S3 : Sharper than the rest; good contrast between sky and ground, and within ground; very friendly. FN 
was not a distracter. Did help. Only would use it at certain times - would like selectable. 

S4: Good for artificial horizon. 

S5: Great terrain portrayal. Good SA. Nice to look at. 

S6: No best feature. 

S7 : Gives him more information than other two textures. 

S8: Needs to have FN; symbology. 

S9: Provides terrain in IMC. 

S 1 0: Better than CCFNs. 

SI 1: Maybe prefer over CC, because of shading (except for CCFN1). 
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SI 2: FN is necessary to get a feel for terrain. 

S 1 3 : FN gives a little definition. 

SI 4: Terrain change in elevation and visual proximity to terrain at horizon. 

S 1 5 : No written c omment. 

S 1 6: Colors are better than CC. 

S 1 7: Can see some of the higher points. 

S18: None. 

S19: Maybe FN helps, but not too much. 

S20: See in the distance with it. Would work fine for shoreline. 

S2 1 : Shading still works. 

S22: Some realistic quality. 

S23: Very crisp and gave good details of surface. FN added little. No change as a function of DEM. 

S24: Contours immediately obvious. 

S25: Florizon. Still have lots of information. 

S26: Color/shading - defines elevations well. 

S27 : Coloring helps elevation. 

PR1 

SI: Cultural cues. Looked close enough, and you could get elevation cues. 

S2: Terrain awareness, avoidance, not distracting, very realistic, shadows. 

S3: Really like. Natural environment. Comfortable. Could pick out highs/lows. 

S4: Transition from OTW to instruments very easy. 

S5 : Good terrain portrayal. Good S A. Provides cultural information. 

S6: Realistic of what you would see in VMC. 

S7: Shows you currently what’ s there. 

S8: Neat; symbology. 

S9: Provides terrain in IMC. Likes topographical representation. 

S10: Incredibly life-like. 

S 1 1 : Everything. 

S12: Almost like looking OTW, don’t feel you’re losing anything; not much interpretation involved. 

S13: Gives a good depth of field. 

SI 4: Terrain elevation at horizon. 

SI 5: Pretty. 

SI 6: Perfect. 

SI 7: Just like outside. Ridges, roads easy to pick out. 

S18: Best one. Gives elevation information and what’s on ground. 

SI 9: Great resolution when low. Landmarks for an emergency situation, gave good information. 

Confidence builder. 

S20: Good detail. 

S21 : Shows what you’re flying over. Shows roads. 

S22: Very close to out the window, clarity was great. 

S23: Looked like a picture. 

S24: Analogous with OTW. Reduces mental workload, increases confidence in device. 

S25: Straight A+. 

S26: Realism - visible landmarks, alternate landing areas. 

S27: Lots. Looks like looking OTW. Good appreciation for heights. 

PRFN1 

SI: Cultural cues. Looked close enough, and you could get elevation cues. FN helped at low altitudes. 

S2: Terrain awareness, avoidance, not distracting, very realistic, shadows. 
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S3: Really like. Natural environment. Comfortable. Could pick out highs/lows. FN helped in this display 
overcome some loss of contouring due to shadows. 

S4: Transition from OTW to instruments very easy. 

S5 : Good terrain portrayal. Good S A. Provides cultural information. 

S6: Realistic of what you would see in VMC. 

S7: Shows you currently what’s there. 

S8: FN; symbology. 

S9: Provides terrain in IMC. 

S10: Incredibly life-like. Has a little more clarification than PR1. 

S 1 1 : Likes better than straight PR. 

S12: FN helps find breaks in terrain. 

S13: Terrain, FN shows valleys. 

S14: Terrain elevation at horizon and close up. 

SI 5: FN - helps make sure colors don’t blend into one. 

SI 6: Perfect. 

SI 7: Just like outside. Ridges, roads easy to pick out. 

SI 8: FN enhances a little. 

SI 9: Great resolution when low. Landmarks for an emergency situation, gave good information. 

Confidence builder. 

S20: FN enhances terrain profile. Easier to see terrain. 

S21: FN helps at higher altitudes. FN in general gives a quick look at what ground is doing without 
having to try and figure it out. Shows roads. 

S22: Like FN; Very close to out the window, clarity was great. 

S23 : Looked like a picture. 

S24: Analogous with OTW. Reduces mental workload, increases confidence in device. 

S25: Straight A+. 

S26: Realism - visible landmarks, alternate landing areas. 

S27: Lots. Looks like looking OTW. Good appreciation for heights. 

PRFN3 

SI: Cultural cues. Looked close enough, and you could get elevation cues. FN helped at low altitudes. 

S2: Terrain awareness, avoidance, not distracting, very realistic, shadows. 

S3: Really like. Natural environment. Comfortable. Could pick out highs/lows. FN helped in this display 
overcome some loss of contouring due to shadows. 

S4: Transition from OTW to instruments very easy. 

S5: Good terrain portrayal. Good SA. Provides cultural information. 

S6: Coloring of terrain. 

S7: Shows you currently what’s there. 

S8: FN; symbology. 

S9: Provides terrain in IMC. 

S10: Incredibly life-like. Has a little more clarification than PR 1. 

SI 1 : Still gave everything that he’s interested in; FN. 

SI 2: Good balance removing clutter without giving up detail. 

S 1 3 : No written c omment. 

S14: Terrain elevation at horizon and close up. 

S15: Not a whole lot good. 

SI 6: Shows lot of terrain. 

SI 7: Terrain still ok, just not as nice as DEMI. 

S 1 8 : No written c omment. 

SI 9: Don’t sense a lot of degradation going from DEMI to DEM3, like he did with EBG. This is really 
good. 
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S20: Can still see terrain. 

S21 : Shows different ground types. FN pretty useful here, because topography doesn’t show much relief. 
Shows roads. 

S22: Photo real and FN. 

S23: Looked like a picture. No changes from 1 to 3DEM. 

S24: Analogous with OTW. Reduces mental workload, increases confidence in device. 

S25: Florizon line is good. 

S26: Realism - visible landmarks, alternate landing areas. 

S27: Still get a feel for terrain. Would be “tickled” with this if he hadn’t seen any of the others. 

PRFN30 

SI: Cultural cues. Terrain cues were missing. 

S2: Terrain awareness, avoidance, not distracting, very realistic, shadows. 

S3: Really like. Natural environment. Comfortable. Could pick out highs/lows. FN helped in this display 
overcome some loss of contouring due to shadows. 

S4: Transition from OTW to instruments very easy. 

S5: Good terrain portrayal. Good SA. Provides cultural information. 

S6: No best feature. 

S7: Shows you currently what’s there. 

S8: FN; symbology. 

S9: Provides terrain in IMC. 

S10: Still good for true emergencies. 

SI 1: FN. 

S12: Some major landmark information (cities). 

S 1 3 : FN gives a little definition. 

SI 4: Terrain elevation at horizon. 

S15: Not a whole lot good. 

SI 6: Pretty color. 

S 1 7: Can see some of the higher points, see a little differentiation between valleys and mountains. 

S 1 8 : No written c omment. 

SI 9: Not like going from 3 to 30 on the others, because of depth you get out of the PR. 

S20: Portrays terrain. 

S21: Shows roads. Shows populations. 

S22: Photo real. 

S23: Looked like a picture. 30DEM was thought to be easier to interpret and not as complex as a scene, 
but still preferred 1 or 3 DEM. 

S24: Analogous with OTW. Reduces mental workload, increases confidence in device. 

S25: Florizon line is good. 

S26: Realism - visible landmarks, alternate landing areas. 

S27: Matches what you see outside. 

19. Please provide what you consider the worst features (points of improvement) of each of the 
following display concepts: 

CCFN1 

SI : Terrain below you - you can’t see it. 

S2: No terrain information. No idea if going into terrain or not. Color. 

S3: Blue sky and brown; lack of features that he can identify. 

S4: Attempts to show height of terrain can give false sense of security. Could see running into terrain 
with this. 

S5: No terrain contrast. 
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S6: The constant color. Red clay may equal flat terrain. 

S7 : Brown - no elevation, just kind of undulates. 

S8: Would add power setting. 

S9: Detail - there could be a little more. 

S10: Zero distinction/contrast 

SI 1 : No shading - shading would help define. 

S12: A lot of thought to interpret FN. 

S13: Doesn’t give you much at all, just showing blue sky and brown ground. 

S14: Poor awareness of terrain elevation close up. 

S15: Only one color. 

S 1 6: Can’t tell terrain details. 

SI 7: Caused dizziness; hard on the eyes. 

SI 8: Doesn’t really show what’s on ground (trees). 

SI 9: FN implementation. 

S20: Lacking. Not multi-color. Flard to read. 

S21 : Flard to tell what’s going on, wavy ground. 

S22: Constant color. Not a good sense of where I was. 

S23: No terrain information. Flard to interpret what the terrain was even though you get some idea of 
contour you don’t get the perspective of what the terrain information means in terms of how high above 
terrain, how flat, can you land there, etc. 

S24: Not intuitive way to show what terrain is doing. Only usable in the lower 40% of screen. 

S25: Doesn’t give a lot of terrain information, for go-arounds this won’t be extremely helpful. 

S26: Too easy to get lost in one color. Flave trouble distinguishing terrain. 

S27: Can’t derive too much information. Not very intuitive. 

CCFN30 

SI: Terrain below you - you can’t see it. And, it’s even worse than DEMI because it implies that 
everything is flat. It’s very misleading. It just doesn’t work. 

S2: No terrain information. No idea if going into terrain or not. Color. 

S3: Blue sky and brown; lack of features that he can identify. Not much to work with. 

S4: Even greater chance of CFIT with the 30 DEM. 

S5: No terrain contrast. 

S6: The constant color. Red clay may equal flat terrain. 

S7: Not as good as CCFNl. 

S8: Would add power setting. FN useless. 

S9: Just a step above SVS baseline. 

S10: Zero distinction/contrast. 

S 1 1 : Losing sharpness. 

S12: Very little information other than flying right towards mountain. 

S13: Doesn’t give you much at all, just showing blue sky and brown ground. 

S 14: Poor awareness of terrain elevation close up. 

SI 5: Little too bland. No good terrain depiction. 

SI 6: All. 

SI 7: Caused dizziness; hard on the eyes. 

S 1 8 : No terrain information. 

SI 9: Might as well be flat. 

S20: False idea of terrain. 

S21 : Hard to tell what’s going on, wavy ground, but worse. 

S22: Resolution loss at DEM30. 
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S23: No terrain information. Hard to interpret what the terrain was even though you get some idea of 
contour you don’t get the perspective of what the terrain information means in terms of how high above 
terrain, how flat, can you land there, etc. 

S24: Not intuitive way to show what terrain is doing. Only usable in the lower 40% of screen. 

S25: No terrain information. 

S26: Too easy to get lost in one color. Have trouble distinguishing terrain. 

S27: Washed out. 

EBG1 

SI : Lack of cultural features (city). 

S2: No cities. Obstacles could be enhanced. 

S3: Nothing. 

S4: Nothing bad. 

S5: Not good at portraying cultural features (cities). 

S6: Does not look like natural terrain. 

S7: No FN. 

S8: Would add power setting. FN - get rid of. 

S9: None. 

S10: Deals only with topography, would like cities, forests, etc. 

SI 1: From roads and symbology, the contrast level is not ideal; harder to pick out man-made obstacles; 
no cities. 

S12: Easy to get too clutter and busy; overwhelms the flight instruments. FN is a non-factor. 

S 1 3 : No written c omment. 

S14: Obstacles need more contrast. 

S 1 5 : Shadowing - a little too dark. 

SI 6: Coloring is not pretty. 

SI 7: Flip colors for shading. 

SI 8: Doesn’t show what’s on the ground (forest and field). 

SI 9: As you get lower, terrain gets darker, and obstacles are hard to see. 

S20: Lower resolution, tiger effect on shadows. 

S21 : Doesn’t depict populated areas. 

S22: Almost real, but not. 

S23: Lacked cultural features. 

S24: Little harder to draw direct correlation to what you see OTW. 

S25: PR spoiled him. 

S26: Lack of realism - trees, power lines, etc. 

S27 : Obstacles are harder to see than in CCFN. 

EBGFN1 

SI : Lack of cultural features (city). 

S2: FN - takes away from seeing obstacles. 

S3: Nothing. 

S4. Nothing bad. 

S5: Not good at portraying cultural features (cities). 

S6: Even less realistic because of FN. 

S7: Maybe needs a legend, color bands absolute or with respect to ownship? What do the colors mean? 
That might be too much information for pilot, though. 

S8: Would add power setting. FN - get rid of. 

S9: None. 

S10: Deals only with topography, would like cities, forests, etc. 
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SI 1: From roads and symbology, the contrast level is not ideal; harder to pick out man-made obstacles; 
no cities. 

SI 2: Easy to get too clutter and busy; overwhelms the symbology. FN is a non- factor. 

S13: Lose a little bit of depth of field compared to PR. 

S14: Obstacles need more contrast. 

S15: FN. 

SI 6: Coloring is not pretty. 

S17: FN. 

SI 8: FN distracting during approach. 

SI 9: FN looks like roads. As you get lower, terrain gets darker, and obstacles are hard to see. 

S20: No written comment. 

S21 : Doesn’t depict populated areas. 

S22: Almost real, but not. 

S23: Lacked cultural features. 

S24: FN distracting, cluttered. 

S25: FN makes too crowded, unrealistic. Interferes with some terrain structures. 

S26: FN. 

S27 : FN does conflict a little. 

EBGFN3 

SI : Lack of cultural features (city). 

S2: FN - takes away from seeing obstacles. Need to focus. 

S3: Hills and valleys not as distinguishable. Kind of blur together, but can pick out some detail. 

S4. Nothing bad. 

S5: Not good at portraying cultural features (cities). 

S6: Resolution - makes it more washed out. 

S7: Isn’t as good as DEMI. 

S8: Would add power setting. FN - get rid of. 

S9: None. 

S10: Adequate resolution, Deals only with topography, would like cities, forests, etc. 

SI 1 : From roads and symbology, the contrast level is not ideal; lose a little clarity, but not overly critical. 
SI 2: None. If had to choose one, this is it. 

S13: Definition is lost. 

S14: Obstacles need more contrast. 

S15: Not enough crispness. Softer. Lose shaipness to hills and valleys. 

SI 6: Not as much depiction of elevation. 

SI 7: FN; little blurry (but workable). 

SI 8: FN shows flatter than actually is, based on coloring information. 

S19: Fairly noticeable difference going from DEMI to DEM3. 

S20: Lacks detail. 

S21 : Things seemed blurred together a little. 

S22: Resolution loss, but not huge. 

S23: Lacked cultural features. 

S24: FN distracting, cluttered. Lower resolution things start to look funny (shading starts to vary). 

S25: Lose a lot of terrain elevation information. 

S26: FN. Lower resolution. 

S27 : Losing a little bit of relief. 

EBGFN30 

SI : Lack of cultural features (city). Flattens things that aren’t flat. 

S2: FN - takes away from seeing obstacles. Need to focus. 
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S3: Could use it to fly, but runs together, a blur. 

S4: Effectiveness of terrain awareness drops off tremendously with DEM30. 

S5: Not good at portraying cultural features (cities). 

S6: Resolution - makes it more washed out, even more so than DEM3. 

S7: Isn’t as good as DEM3. 

S8: Would add power setting. Fuzziness. 

S9: Losing some detail. 

S 1 0: Marginal resolution for true terrain avoidance. 

S 1 1 : From roads and symbology, the contrast level is not ideal; lower resolution, don’t have the sense of 
security. 

S12: Details of terrain washed out; less of a 3-D feel. 

S 1 3 : Not much terrain at all. 

S14: Obstacles need more contrast and close terrain becomes fuzzy. 

S 1 5 : Really bland. All meshes together. FN. 

SI 6: Flat. 

SI 7: Blurriness and fuzziness distracting; FN. 

S18: Too flat. 

SI 9: Flattens so much that shading doesn’t make sense. 

S20: Lacking terrain. 

S21 : FN gets lost and doesn’t do much at all (tends to disappear because ground looks flat). 

S22: Resolution loss is big. 

S23: Lacked cultural features. Lose detail in terrain. 

S24: FN distracting, cluttered. Lower resolution things start to look funny (shading starts to vary). Loses 
even more features than the DEM3, but still has shading for those features. If elevation map is low 
resolution, make shading low resolution so it matches. 

S25 : Up front, lose all sense for height. FN even worse. 

S26: FN. Lower resolution. 

S27: Too flat to have any appreciation. 

PR1 

SI : Cluttered, makes difficult to discern objects. 

S2: Color of obstacles. 

S3: So natural, green dark shadows made more difficult to pick out undulations. 

S4: Descending, it’s difficult to tell different terrain heights. 

S5: Not as good at terrain contrast. Couldn’t pick out valleys and ridges as well. 

S6: Inability to differentiate the dark colors. 

S7: Isn’t showing elevation, just showing you what’s currently there. 

S8: Would add power setting. Softened terrain features. 

S9: None. 

S10: Sometimes had to think: close versus far on mountain. Almost too realistic. 

S 1 1 : No real bad thing. 

S12: Some of terrain detail can get lost as background colors. 

S 1 3 : No written c omment. 

S14: Obstacles need more contrast. And terrain requires more time to comprehend. 

S 1 5 : Doesn’t give topography outline. Definition. 

S16: None. 

SI 7: None. 

SI 8: Landmarks (roads/waterways) harder to pick up. 

S19: Colors hide towers. 

S20: Have to look at a little longer to pick out details, sometimes a question of what’s what. 

S21: Distracting, tend to focus head down, due to “awe” value (fixation). 
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S22: None. 

S23: Concerned about trust and complacency because it was too real looking. May trust it too much and 
forget that it should be cross-checked. 

S24: Maybe a little harder to tell elevations than with EBG where you have texturing. 

S25: None. Might be too much information in certain cases. EBG1 will do the job for him. 

S26: Can’t distinguish elevations as easily as EBG. 

S27 : Obstacles are a little harder to find. 

PRFN1 

SI: Cluttered, makes difficult to discern objects. 

S2: FN. 

S3: So natural, green dark shadows made more difficult to pick out undulations. 

S4: Descending, it’s difficult to tell different terrain heights. 

S5: Not as good at terrain contrast. Couldn’t pick out valleys and ridges as well. 

S6: FN. 

S7: Isn’t showing elevation, just showing you what’s currently there. 

S8: Would add power setting. Softened terrain features. 

S9: None. 

S10: Grid - contrast could be better. 

SI 1 : None. 

S12: Amount of information tends to get too cluttered. There is overlap. 

S13: None. 

SI 4: Obstacles need more contrast. And terrain requires more time to comprehend. 

S 1 5 : FN is spaced too far apart. 

S16: None. 

SI 7: FN. 

SI 8: FN could distract on approach. 

SI 9: FN adds confusion to a good display. 

S20: No written comment. 

S21: Distracting, tend to focus head down, due to “awe” value (fixation). 

S22: None. 

S23: Concerned about trust and complacency because it was too real looking. May trust it too much and 
forget that it should be cross-checked. 

S24: FN more distracting than helpful. 

S25: FN- leave it out. 

S26: FN. 

S27 : FN doesn’t hurt, but it doesn’t add either. 

PRFN3 

SI : Cluttered, makes difficult to discern objects. 

S2: FN, and need to focus. 

S3 : Starting to blur. 

S4: Descending, it’s difficult to tell different terrain heights. 

S5: Not as good at terrain contrast. Couldn’t pick out valleys and ridges as well. 

S6: Resolution - makes terrain washed out. 

S7: Isn’t showing elevation, just showing you what’s currently there. 

S8: Would add power setting. Softened terrain features. 

S9: None. 

S10: Good resolution. 

S 1 1 : Fosing some quality of picture. 

S12: Not much absolute elevation information (low versus high ground). 
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S 1 3 : Definition not quite as good. 

SI 4: Obstacles need more contrast. And terrain requires more time to comprehend. 

SI 5: Blurry. 

SI 6: Not as detailed as DEMI. 

SI 7: FN. 

S18: Would be better without FN. Could get 3-D depiction without FN. Would want resolution to be 
higher. 

SI 9: FN adds confusion to a good display. 

S20: Fower detail. 

S21 : Claiming photo realistic, but hides things (misleading). 

S22: Slight resolution loss. 

S23: Concerned about trust and complacency because it was too real looking. May trust it too much and 
forget that it should be cross-checked. 

S24: Starts to get a lot harder to tell contours. EBGFN3 is a bit easier. 

S25 : Things really start deteriorating here, FN more irritating. 

S26: FN. Fower resolution. 

S27: Fose a little resolution. 

PRFN30 

SI: Cluttered, makes difficult to discern objects. Flattens terrain. 

S2: FN, and need to focus - very considerable. Terrain awareness goes down a bit, looks flat. 

S3: Even blunder than PRFN3. Not a lot to work with as far as what’s a hill, gradual creep versus steep. 
S4: Terrain awareness drops off - resolution is not high enough. 

S5: Not as good at terrain contrast. Couldn’t pick out valleys and ridges as well. 

S6: Resolution - makes terrain really washed out. 

S7: Isn’t showing elevation, just showing you what’s currently there. 

S8: Would add power setting. Fuzziness. 

S9: Fosing some detail. 

S10: Marginal resolution. 

S 1 1 : Everything looks smooth, flat; can lose FN overlay in the distance. 

S12: 3-D gets washed out, colors blend. Interpreting FN. 

S 1 3 : Not much terrain at all. 

S14: Obstacles need more contrast. And terrain requires more time to comprehend. Close to terrain is 
fuzzy. 

S 1 5 : Blurry - worse than DEM3 . 

S 1 6 : T errain doesn’t show. 

S 1 7: Blurriness and FN. 

S18: Too flat. 

SI 9: FN adds confusion to a good display. 

S20: Facks 3-D effect, lacking detail. 

S21 : Pretty dangerous. Poor depiction of what’s on ground. 

S22: Big resolution loss. 

S23: Concerned about trust and complacency because it was too real looking. May trust it too much and 
forget that it should be cross-checked. 

S24: Starts to get a lot harder to tell contours, particularly so. Actually didn’t like PRFN30 for some 
tasks. 

S25 : Things really start deteriorating here, FN more irritating. 

S26: FN. Fower resolution. 

S27 : Approaching too flat to be useful. 
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20. Please provide an estimate of likelihood that a pilot would encounter a controlled flight into 
terrain situation while using each of these display concepts. Use 0 to 10 ranking with 0 being 100% 
never and 10 100% likelihood of encountering CFIT (others are that percentage, i.e., 2 = 20%). 
What is the reason for that estimation? 

Most subjects answered this question in a relative fashion - comparing all SVS concepts to the 
baselines. 


Subject 

BSBG 

BRD 

CCFN1 

CCFN30 

EBG1 

EBGFN1 

EBGFN3 

EBGFN3 

0 

PR1 

PRFN1 

PRFN3 

PRFN30 

1 

3 


3 

3 

1 

1 

1 

2 

1 

1 

1 

2 

2 

10 


9 

9 

0 

2 

3 

6 

0 

0.1 

2 

3 

3 

8 


6 

7 

2.5 

2.5 

3 

5 

3 

3 

4 

5 

4 


10 

6 

8 

0 

0 

1 

7 

1 

1 

4 

7 

5 

4 


3 

3 

2 

2 

2 

2 

2 

2 

2 

2 

6 


3 

1.5 

1.5 

0.5 

0.5 

1 

1.5 

1 

1.5 

1.5 

1.5 

7 

10 


9 

9 

1 

1 

1 

1 

3 

3 

3 

3 

8 


10 

3 

4 

1 

1 

1 

2 

2 

2 

2 

3 

9 

7 


4 

5 

1 

1 

1 

2 

1 

1 

1 

2 

10 


10 

3 

4 

1 

1 

2 

3 

2 

1 

2 

3 

11 

10 


2 

4 

1 

1 

2 

3 

0.5 

0.5 

1 

2 

12 


9 

3 

5 

1 

0.5 

0.5 

1 

1.5 

1 

0.5 

1 

13 

10 


5 

7 

3 

3 

5 

7 

2 

2 

4 

7 

14 


10 

5 

5 

2 

1 

2 

3 

2 

2 

4 

5 

15 

5 


4 

5 

2 

2 

3 

4 

2 

2 

3 

4 

16 

10 


9 

10 

1 

1 

2 

5.5 

0 

0 

2 

3.5 

17 

10 


10 

10 

0 

1 

3 

7 

0 

0 

2 

5 

18 


9 

2 

7 

0 

0 

3 

7 

0 

0 

4 

7 

19 


4 

3 

3 

2 

2 

2 

3 

1 

1 

1 

1 

20 


10 

4 

5 

2 

1 

3 

4 

2 

1 

3 

4 

21 


5 

3 

3 

1 

1 

2 

2 

1 

1 

2 

2 

22 


9 

5 

5 

3 

3 

3 

3 

2 

2 

2 

2 

23 

10 


9 

10 

4 

4 

4.5 

5 

3 

3 

3 

4 

24 


10 

5 

7 

3 

3 

5 

6 

3 

3 

5 

6 

25 


7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

26 

5 


4 

4 

1 

1 

2 

2 

1 

1 

2 

2 

27 


9 

8 

8 

3 

3 

4 

6 

2 

2 

2 

5 

Max 

10 

10 

10 

10 

4 

4 

5 

7 

3 

3 

5 

7 

Min 

3 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Ave 

7.9 

8.6 

4.8 

5.6 

1.4 

1.5 

2.3 

3.7 

1.4 

1.4 

2.3 

3.4 

STDV 

2.7 

2.5 

2.7 

2.7 

1.1 

1.1 

1.3 

2.1 

1.0 

1.0 

1.3 

1.9 
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SI: No written comments 

S2: BSBGBL - if you’re not studying altitude and map, could be in trouble. CCFN1 and CCFN30 - not 
much better than BSBGBL. EBG1 - would not fly into terrain with this one. Would see terrain coming at 
you. EBGFN1 - would be distracted a bit by FN. EBGFN3 - focus is an issue, and might not actually see 
the mountain. EBGFN30 - You may have no idea that it is a mountain, or that it’s coming at you. PR1 - 
would know if you’re flying into terrain. PRFN1 - So realistic. PRFN3 - FN distracts. PRFN30 - still 
realistic, FN looks flat, though. 

S3: No written comments. 

S4: In general, on approach, if he was off GS, with the lower resolutions, felt less of a need to get back 
on GS. With BRD, entering IMG, can see getting disoriented. CCFN1 - attitude would help VFR pilot 
flying into IMC, but would tend to fly closer to terrain than is safe. CCFN30 - Resolution would be 
trusted more than should, so not as effective. EBG1 - by example, felt it gave him a heads-up that he was 
too low 1 - 1.5 minutes in advance. Same with EBGFN1. EBGFN3 - some loss of resolution. EBGFN30 - 
resolution is so much lower. PR1, PRFN1 - Nose-down, peaks kind of blend together. PRFN3 - 
combining lower resolution with lack of color contrasting. PRFN30 - resolution is too low. 

S5: Gave ratings based on using the MX20 in conjunction with the SVS. Not 0’s or l’s because other 
distractions are always there, can have misinteipretations, etc. Never will be able to prevent, but certainly 
will be able to diminish likelihood. 

S6: Answered this question with the thought of an average pilot in a GA aircraft. Feels that the answer 
would NEVER be 1 00% and NEVER be 0%. 

S7: Helping pilot to avoid CFIT is the SVS diplays’ major contribution. 

S8: No written comments. 

S9: Answered question with the assumption that the pilot encountered IMC. Feels CCFNs would be 
better than the SVS baseline, because it has topography and towers. Also, the EBGFN30 and PRFN30 
ra nk a little lower than the other EBGs and PRs because you start to lose a little with resolution. 

S10: All of the SVS concepts would make a big difference over standard gauges. Standard gauges, could 
easily see pilots (based on history) encountering CFIT. 

S 1 1 : Baseline SVS does not tell you much of anything in terms of terrain. Between CCFNl and 
CCFN30 - resolution does matter. EBG1 and EBGFN1 - have a good idea that something is right there. 
EBGFN3 - losing some terrain through lower resolution. EBGFN30 - even more so. PR1 and PRFN1 - 
to a certain extent, need to know your instruments. Would need some level of instrument proficiency. 
PRFN3 - FN helping some, PR helps a lot. PRFN30 - even more reliant on instrument skill. Does give 
more horizon than traditional gauges. 

S12: Ranking these relatively. For EBGFN30, the FN makes up for detail lost. 

S13: At 100%, the baseline SVS is just a step above standard gauges, where the standard gauges might 
be 120%. 

S 1 4 : No written c omments . 

S 1 5 : There is still always a chance that it will happen. 

S 1 6 : No written c omments . 

SI 7: EBG1 - details are just so nice. Rated EBGFN1 a little lower, due to FN being a distraction. Rated 
EBGFN30 even lower, because resolution makes a difference, here. Rated PR1 and PRFN1 the same, 
because even though FN is distracting, realism wins. 

SI 8: Answered this question as if he had no MX20 information. With standard gauges, feels there is a 
pretty good chance of encountering CFIT. Feels that CCFN30, the likelihood would be high, due to lack 
of resolution. EBG1, there is no doubt it would help. EBGFN3 - still get the terrain depiction, so 
likelihood would go up, but only slightly. 

SI 9: Accident statistics are a probably a little high, so rated BRD a 4 (instead of higher). For all the rest 
of the SVS displays, you’ll have VV that tells you you’re going to hit terrain. CCFNl - don’t have a 
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good feel for terrain. For EBGFN30 - resolution degrades just enough to impact effectiveness. And, for 
all PRs, would be hard pressed to fly into terrain with any of them. 

S20: Waterways are very important to him, and how each concept display water played into how he rated 
each concept. Fie also rated each concept as if he were flying in Alaska. 

S21 : No written comments. 

S22: No written comments. 

S23: No written comments. 

S24: BRD - obviously the worst. CCFN1 - DEMI will be better than lower resolution. EBGFN1 - FN 
doesn’t make a difference. PR1 - likes it better than EBG, but both give equally good SA close to terrain. 
S25: Fle’s moderately comfortable with standard gauges. For pretty much all of the SVS concepts, he 
feels confident he won’t end up in the terrain. If he stays alert and pays attention. Flowever, gross pilot 
error may induce CFIT. 

S26: Being able to see ground is real comfort. Didn’t consider the MX20 when giving his answers. If he 
cross-reference with the MX20, these percentages would get better. 

S27: Standard gauges - this has happened before and will happen again. CCFNs - doesn’t rule CFIT 
out, can still happen, because can’t tell what terrain is doing on either. EBG1 - has good information. 
EBGFN3 - takes a little longer to process. EBGFN30 - have to get close before you realize. PR1 - 
would have to work to actually have a CFIT with this one. Ranked PRFN3 the same, because he flew it 
on the rare event, and did fine - gave him a good feel. PRFN30 - foreground is a little flat. 
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21. Please provide an estimate of likelihood that a pilot would encounter the onset of a low 
visibility loss of control situation while using each of these display concepts. Use 0 to 10 ranking 
with 0 being 100% never and 10 100% likelihood of encountering LVLoC (others are that 
percentage, i.e., 2 = 20%). What is the reason for that estimation? 

Most subjects answered this question in a relative fashion - comparing all SVS concepts to the 
baselines. 


Subject 

BSBG 

BRD 

CCFN1 

CCFN30 

EBG1 

EBGFN1 

EBGFN3 

EBGFN3 

0 

PR1 

PRFN1 

PRFN3 

PRFN30 

1 

0.1 


0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

2 

8 


8 

8 

0 

0.1 

0.2 

4 

0 

0.1 

0.2 

4 

3 

8.5 


6 

7 

2.5 

2.5 

3 

5 

2.5 

2.5 

4 

5 

4 


10 

6 

8 

0 

0 

1 

7 

1 

1 

4 

7 

5 

3 


3 

3 

2 

2 

2 

2 

2 

2 

2 

2 

6 













7 

10 


5 

5 

1 

1 

1 

1 

1 

1 

1 

1 

8 


10 

3 

3 

1 

1 

3 

3 

3 

3 

3 

3 

9 

7 


4 

5 

1 

1 

1 

2 

1 

1 

1 

2 

10 


10 

4 

5 

2 

2 

3 

4 

3 

2 

3 

4 

11 

2.5 


2 

2 

1 

1 

1 

1 

0.5 

0.5 

0.5 

0.5 

12 


8 

4 

4 

2 

2 

1.5 

2 

1 

1 

1 

2 

13 

10 


5 

7 

3 

3 

5 

7 

2 

2 

4 

7 

14 


10 

4 

5 

1 

1 

2 

3 

1 

1 

2 

3 

15 

1 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

16 

0.5 


0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

17 

10 


10 

10 

0 

0 

1 

3 

0 

0 

1 

3 

18 


9 

2 

3 

2 

2 

3 

3 

2 

2 

3 

3 

19 


5 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

20 


2 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

21 


6 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

22 


9 

5 

5 

3 

3 

3 

3 

2 

2 

2 

2 

23 

10 


9 

10 

7 

7 

7 

7 

5 

5 

5 

5 

24 


10 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

25 


7 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

26 

5 


4 

4 

1 

1 

1 

2 

1 

1 

1 

2 

27 


9 

4 

5 

3 

3 

4 

5 

2 

2 

3 

4 

Max 

10 

10 

10 

10 

7 

7 

7 

7 

5 

5 

5 

7 

Min 

0.1 

2 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

Ave 

5.8 

8.1 

3.9 

4.3 

1.7 

1.7 

2.1 

3.0 

1.6 

1.6 

2.1 

2.8 

STDV 

4.0 

2.5 

2.5 

2.8 

1.5 

1.5 

1.6 

2.0 

1.2 

1.2 

1.4 

1.9 


SI: All of these displays are easy to interpret - big horizon line. Terrain may hurt, when have steep nose 
angles, and display becomes filled with terrain. SVS BL is simpler to interpret, but doesn’t include 
ground rush or drift cues. All displays have sufficient information to prevent LVLOC. 
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S2: BSBGBL, CCFN1, CCFN30 - horizon more of a factor/help, because you’re pretty much strictly 
gauges on all three of these. EBG1 - you’ll have your horizon and terrain awareness. EBGFN1 - FN is a 
bit of distraction. EBGFN3 - FN, and blurriness even more so. EBGFN30 - blurriness really throws you 
off. PR1 - would not have a problem, very realistic. PRFN1 - FN distracts slightly. PRFN3 - extremely 
realistic, nice horizon, not too distracting. PRFN30 - fuzziness distracting. 

S3: No written comments. 

S4: Feels numerical answers would be same as in the CFIT case. 

S5: Nice big horizon line will give you some input, and showing ground may not give much more input. 
S6: Terrain really won’t make a difference in this case because you have a big horizon line and a big 
display for all of them (except standard gauges). BRD - high probability. CCFN1, CCFN30, EBGFN1, 
EBGFN3, EBGFN30, PRFN1, PRFN3, PRFN30 - low probability. EBG1 and PR1 - even lower 
probability, because more detail may help a little. 

S7: Being able to see the terrain (all concepts) gives you a warm fuzzy. 

S8: All would absolutely help out! 

S9: Thinks that the answers would be the same as for CFIT. Hard for him to determine, because he 
wasn’t experiencing it, in terms of spatial disorientation. 

S10: Would make a difference if the pilot was IFR or VFR rated. Instruments and gear will only do so 
much, the rest is up to the pilot. 

SI 1: With minimal training on instruments, would be able to use SVS easily. Baseline SVS is easier to 
interpret than standard gauges. 

S12: Again, ranking relatively. CCFN1 and CCFN30 - resolution doesn’t matter. EBG1 and EBGFN1 - 
FN doesn’t matter. Level of detail sometimes got a little disorienting. PR1 and PRFN1 - feels 
comfortable with no interpretation. 

S 1 3 : Same as CFIT answer. 

S 1 4 : No written c omments . 

SI 5: Pilots can still be dumb and still do it. All displays give enough information to rank the same. 

SI 6: All of these displays provide enough information to prevent LVLOC. 

SI 7: EBG1 - so nice having terrain under there. EBGFN1 - blue sky, terrain, would be so easy to keep 
horizon in sight. EBGFN3 - having the picture of terrain helps. EBGFN30 and PRFN30 - vertigo added 
to blurriness, would not be real helpful. 

S18: Terrain doesn’t make as much of a difference, here. Resolution does enhance ability to determine 
where you are. 

SI 9: Can tell wings level just by looking at the SVS displays. They are more like what you see OTW. 

S20: Ranked all SVS concepts the same - at straight and level, terrain doesn’t really matter. However, 
ranked BRD better than the rest, because when it come to controlling the aircraft, he’s familiar with the 
dials. So, he thi nks it’s better, right now. Maybe once he became more familiar with it, it would change. 
How does this generate in a spin? What would the update rate be? 

S21: People seem to do this all of the time. With SVS displays, can always tell what’s up and what’s 
down. 

S22: No written comments. 

S23: No written comments. 

S24: BRD - we know LVLOC happens with this one. All SVS displays give the horizon line and pitch 
ladder - a little easier to see gauges. Just fact that you have terrain in the picture will help you keep your 
bearing. All have very well defined horizon. 

S25: Even in clouds, still have clean view on the PFD. These concepts make him more than comfortable 
flying in low visibility. 

S26: With the baseline SVS, might feel frustration and anxiety, so gave a lower rating. It would probably 
be all right, if pilot had some instrument training. With the EBGs and PRs, they were all realistic and 
gave ground reference. Being able to see ground is real comfort. 
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S27: CCFN1 - terrain does play into it, but symbology is more important. CCFN30 is rated lower due to 
washed out effects. EBG1 and EBGFN1 - would like landmarks. EBGFN3 - a little degraded. PR1 - 
see what you need to see. PRFN3 - still pretty good. PRFN30 - losing terrain. 

22. Please provide any additional comments that will help us in evaluating the display concepts. 

SI: N/A 

S2: MX20 not very useful at end of approach, because it only shows red (in terrain awareness mode) once 
you get down to a certain altitude. CCFN30 tunnel versus no tunnel - tunnel is saving grace, especially if 
had NO instrument training. Suggests putting heading number above VV when VV occludes it. 

S3: Suggests adding practice time for the second day. Also, maybe add a run with weather during 
training. Would like a little more notice when it’s time to “rest” for the physiological data. 

S4: In his opinion, the only display concepts to consider would be EBG1, EBG3, PR1, PR3. Would not 
recommend DEM30 - false sense of security. FN doesn’t seem to help, even hinders. So, would drop FN 
- clutters display. Tunnels - found very helpful, reduced workload. Maybe make the width of the tunnel 
selectable. Standard gauges versus CCFN - would be a lot easier to become inverted on basic gauges. 
PR is nice for transitioning from OTW to HDD, especially if the weather is clear. Really likes 
acceleration arrow, VV, and heading indicator (especially effective). CCFN30 and CCFNl - did not like, 
but as far as general safety, would take of standard gauges, because they are a huge improvement. 

S5: CFIT is a much better example of this than LVLOC. Tremendous benefits in CFIT, but too 

experienced to comment on LVLOC. Suggests motion simulator to help evaluation these concepts with 
respect to LVLOC. Also, these concepts help in navigation and staying clear of places you don’t want to 
be. All terrain concepts are nice. Suggested taking out turbulence -too big of a transport delay. Input 
versus response is too long. Turbulence is not quite realistic - takes away from looking at the display. 
Make easier to fly so pilots can evaluate SVS. 

S6: Suggested doing a little more flying closer to the terrain. Does NOT like waterline - would get crab 
angle from magnetic heading and VV. Scan without tunnel is very difficult. 

S7: During the experiment, having more non-tunnel scenarios would have been helpful. Maybe put SVS 
concept on a HUD. In terms of symbology, suggests adding a stall indicator or angle of attack meter. 

S8: Too many concepts to look at - got confusing. Seven concepts or less would have been more 
manageable. DEMI and 3 - not enough of a difference, so we could have done without the DEMI. Fly 
SVS baseline at the beginning and at the end - he was flying numbers, so ignored terrain. Not until you 
flew rare event did the terrain make a difference. 

S9: If cost is not a factor, use best resolution for each of the terrain textures. Can leave FN on there, 
doesn’t really bother him. If he had to choose 6 to take to flight, he’d pick: PR1, EBG1, CCFNl, 
EBGFN1, PRFN1, CCFN3. Would like to see implementation of audio warning for SVS terrain. 

S10: Had fun. Likes VV. Would want 6-pack redundant gauges. Likes FOV choices - would prefer to 
toggle down versus up, though. Also, likes the FOV yoke switch (as opposed to the knob on panel). 

SI 1 : FOV - didn’t like left-hand knob; did like toggle button on yoke; would be nice to have up and 
down capability on yoke. Was overloaded at end of experiment. Realistically, had a hard time with depth 
perception. Lags and glitches in the screen. 

S12: Transition OTW to HD - PR, no interpretation; the other two required interpretation. Suggested 
improvements: 1) point of view control - for large cross-wind conditions, to avoid VV down in a comer; 
2) move horizon line up so that can see more terrain below you, in en route; 3) Adjusting terrain view 
(FOV) without changing instrument FOV so you could go to 90 without losing controllability - decouple 
somehow; 4) Approach - decision height indicator would be nice, to cue you to look OTW when you’re 
at the DH. 

S13: Get rid of dogbones when you pick up the GS/Loc. Make the center of the GS/Loc scale a different 
color. VSI - give it more tics, and move it over. When making turn, the pitch ladder covers the roll 
scale. 

S14: Without boxes: 
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• Used to seeing heading displayed on an arc with digital readout. I confused this digital heading 
with your roll scale. 

• Would prefer that heading scale remained the same in all vies. This may not be possible. 

• 1 like having pitch scale in center of display around pitch reference symbol. Pitch reference 
symbol interferes with VV. VV is more important. 

• Horizon is easily seen. 

• Airspeed and altitude easy to comprehend, but pilot must have active scan, as much as with round 
dials. 

With boxes: 

• This solves the high workload scan problem. Heading, altitude, GDI, and GS displays do not 
have to be in the scan unless you want to verify what the boxes are telling you and where they are 
taking you. 

• The scan now is the boxes, airspeed, and terrain, and you have more time to view the MFD and 
manage other airplane systems and talk to ATC. 

• When en route and even early on approach, it should not be necessary to always fly inside the 
boxes, but simply near them. 320’x400’ accuracy is not needed here, but the awareness of the 
course presented by the boxes is very easy to see and comprehend. How far away from the boxes 
is acceptable is a question I can’t answer. To be one dot off centerline or one dot high or low on 
a GS is I think the normal tolerance. This is well beyond the boxes. 

I’ve really enjoyed and learned a lot from this. Hope to see you again. 

S15: Really likes symbology. Do not like waterline - would like to have screen centered on VV. PR is 
pretty, but with the coloring on the EBG, EBG is his preference. Speed and altitude tapes get a little 
confusing. 

SI 6: PR - will sell. Everything is there. Clearly depicts terrain and altitude. Pilots won’t run into 
anything (terrain, obstacles, etc.). Symbology - a tack indicator would be good. System - wonderful 
idea, PR with or without tunnel; zero/zero visibility, tunnel makes it easier. 

SI 7: Symbology - a lot of information on the display. Altitude/VSI combination was cluttered. Maybe 
color VS1 arrow and digital read-out on altitude. Tunnel was great. For the 20° bank, would be nice to 
have an aiming point. Really likes VV. Overall, a nice system. 

SI 8: Slip indicator on VV is a great tool. Throttle caret - putting it where you need it really helps a lot. 
Outer marker and middle marker - suggests using different color boxes to indicate where you are. 

S19: PR3 (without FN) would be great for GA, if there is a significant cost difference for DEMI. Also, 
EBG is very good at DEMI, but not at DEM30 - it seems that the quality of EBG degrades fast (faster 
than the PR), with the lower resolutions. How do we tweak this thing in terms of symbology? Has to be 
a better way for steering guidance, on tunnel. Maybe ghost aircraft or traditional flight director or 
pathway. Bank angle on roll scale - rate seemed twice that of horizon line. Concentrating on VV cluster, 
not looking at the roll pointer. Rental GA’s - should NOT have sky pointers. Maybe should have type- 
rating for that. 

S20: All SVS displays have good aircraft controls. Symbology - likes arcs on the speed tape. A FOV 
indicator would be nice. CDI - make box and diamonds different colors - also, maybe make it red at full 
deflection. Play around with colors on symbology. Maybe keep magenta runway. 

S21: Didn’t like flag for slip indicator - didn’t like sensitivity or location. VS1 - didn’t like, hard to 
judge how quickly it’s rate increased, cluttered with altitude tape and read-out. Liked airspeed. Missed 
having tachometer. Liked acceleration arrow. Suggested putting waypoints on the PFD - to depict when 
you’re there (pops up within two minutes of getting there). 

S22: Liked options and cross-testing. Explore failure testing and reversionary modes. 

S23: No written comments. 

S24: Likes instruments - easy to comprehend. Terrain representations are very valuable, particularly PR. 
Add navigation to HDD (like flag showing where airport is). Really likes VV cluster. 
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S25: Would like to have the option to tilt the display. Why don’t we have this yet? He missed motion in 
the sim. Display is small, tires his eyes - can it get bigger? 

S26: No written comments. 

S27: Possibly a 4-way toggle on yoke so can go straight to one FOV. Would like capability of turning 
off or adjusting FN. Maybe vertically exaggerate with the DEM30s would help with the terrain elevation. 
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Appendix E: Rare Event Data 


The rare-event results were grouped into four categories. Category A contains those subjects who were 
very aware of their surroundings, and indicated well in advance that they felt there was something amiss. 
The subject pilots in Category A were judged to be in a safe position and had adequate time to maneuver 
to steer clear of terrain. Subject pilots who identified that something was amiss, but did so either within 
500 ft of impact were placed in Category B. Category B was designated CFIT “incidents” - not 
necessarily a crash, but definitely a safety-of-flight concern. Category C indicated subject pilots who 
identified that something was amiss, but were first cued by OTW information (visibility on the OTW 
display was one statute mile), rather than their instrument displays. And, finally, Category D represented 
subject pilots who actually flew into terrain. Table FI illustrates the individual results for each subject 
pilot. 


Table FI. Rare event Data 


Subject Pilot 

Display 

Concept 

Pilot 

Qualification 

AGL at Time of 
Recognition, ft 

Category 

1 

CCFN1 

Test Pilot 

1810.5 

A 

2 

PRFN3 

VFR 

1342.1 

A 

3 

EBGFN30 

VFR 

1038.7 

A 

4 

EBG1 

IFR 

308.7 

B 

5 

CCFN30 

Test Pilot 

1032.6 

A 

6 

PRFN1 

Test Pilot 

90.1 

B 

7 

EBGFN3 

VFR 

1386.1 

A 

8 

EBGFN1 

VFR 

2343.8 

A 

9 

PRFN30 

VFR 

1168.3 

A 

10 

PR1 

IFR 

667.1 

A 

11 

CCFN30 

Capstone 

1477.0 

A 

12 

EBG1 

IFR 

1274.0 

A 

13 

EBGFN1 

Capstone 

177.7 

B 

14 

PR1 

Test Pilot 

1058.0 

A 

15 

PRFN1 

VFR 

1243.2 

A 

16 

EBGFN3 

IFR 

941.8 

A 

17 

PRFN3 

IFR 

864.3 

A 

18 

EBGFN30 

VFR 

777.6 

A 

19 

PRFN30 

IFR 

1641.5 

A 

20 

CCFN1 

Capstone 

1091.1 

C 

21 

CCFN30 

VFR 

1264.4 

C 

22 

EBG1 

VFR 

-130.2 

D 

23 

EBGFN1 

VFR 

1135.0 

C 

24 

PR1 

VFR 

792.3 

C 

25 

PRFN1 

VFR 

-45.0 

D 

26 

EBGFN3 

VFR 

633.9 

A 

27 

PR3 

VFR 

1357.7 

A 
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